[ home / rules / faq ] [ overboard / sfw / alt ] [ leftypol / edu / labor / siberia / lgbt / latam / hobby / tech / games / anime / music / draw / AKM ] [ meta ] [ wiki / shop / tv / tiktok / twitter / patreon ] [ GET / ref / marx / booru ]

/tech/ - Technology

"Technology reveals the active relation of man to nature" - Karl Marx
Name
Options
Subject
Comment
Flag
File
Embed
Password(For file deletion.)
Not reporting is bourgeois
[ Return / Go to bottom ]

File: 1734318091170-0.png (50.35 KB, 1920x1080, 2024-12-15-205512_1920x108….png)

File: 1734318091171-1.png (76.78 KB, 1920x1080, 2024-12-15-205932_1920x108….png)

 

This is the next installment of threads in my project log series since the imageboard was completed >>27187, and ran out of motivation for the calendar-based forum >>27553, leaving it as merely a (relatively complete) calendar. The idea this time around is to make something of a GNU Emacs replacement, but broken up into a number of tightly coupled smaller packages. Namely the approach is going to be "frames" over "windows" where windows are controlled by a programmable tiling window manager and run in Xephr for individuals who don't want to replace their window manager. Further there will be a command processor for mapping keybindings to functions and command names to functions. Next there will be a prompt package which will be responsible for prompting the user, more likely than not this will be either at the cursor position or in the center of the display. Lastly, the text editor itself will be responsible for very little beyond saving and reloading state, and rendering the text buffer; these two functionality will likely also be split into separate packages.

Presently, the implementation is in C, Xlib, Cairo, and Pango but is little more than a demo to edit and render text taken from various tutorials glued together only started yesterday afternoon. It is however already capable of indexing in text based on pointer events, so it seems the stack will be powerful enough to write the gui. Am likely to start with morphing the present program into the prompt package and once have the basics of the centered view created (this would be something akin to dmenu) am likely to begin rewriting in Common Lisp. Don't yet know how to handle the interprocess communication ideally it would be bidirectoinal and work well with programs not written in Common Lisp, though this latter point isn't the biggest priority. Getting the interprogram communication right is going to be critical for having the environment work well overall. Anyway that's all for now.

File: 1734382637606.png (83.66 KB, 1920x1080, 2024-12-16-151238_1920x108….png)

>>27796
>Presently, the implementation is in C, Xlib, Cairo, and Pango but is little more than a demo to edit and render text taken from various tutorials glued together.
The demo was extended a little today with an added keybinding for escape and keyboard based navigation for a single line. There was also some refactoring done, switching from calloc to static buffer sizes, and a simple bug fixes to do with using memcpy instead of memmove. Lastly a visual cursor was added.

The next two major hurdles are going to be positioning the window, and translating the Cairo surface to show only the correct portion of the Pango layout. The simplest way to implement the latter can think of at the moment is to maintain a second cairo surface large enough to fit the full Pango layout and to copy rectangles from the large surface to the xlib surface. Haven't implemented let alone instrumented this yet, but it sounds like for large buffers it would be slow. An optimization would be to perform these same functions but only render N lines at a time where N is computed from the window height divided by the line-spacing. Unfortunately the minimum, and maximum font-width is not available to allow for a similar optimization in width.

Smaller tasks that might be completed before these others is configuring various attributes of the windows such as hinting for the title, grabbing the mouse and keyboard, and accepting clipboard usage. In fact it may even be reasonable to complete some of this tonight.

File: 1734395562687.png (970.38 KB, 1000x718, thanks4sharing.png)

This looks really cool anon

>>27807
What a creepy post.

>>27805
>minimum, and maximum font-width is not available
I haven't used pango in a long time, but with monospace fonts the minimum and maximum should be the same, besides the rare exception, which can be ignored for the sake of simplicity. even if the api doesn't give you a width, I think you can take an arbitrary glyph or whatever it was called and take the width from there. as long as it is a monospace font you can pretend this isn't a terrible idea
if it isn't monospace, I'm sure the api must have a call to measure without actually rendering

>27812
>if it isn't monospace, I'm sure the api must have a call to measure without actually rendering
Haven't implemented it yet, but think you're right. Think you can create a layout with just the relevant lines without a surface to render on and then get the pixel extents. This would results in renders with everything correct except for the exceptional cases where the line-height isn't respected. Another optimization might be to force wrapping if lines are too excessively long (e.g. thousands of characters), though this would be unlikely to be implemented initially.

>>27805
>Smaller tasks that might be completed before these others is configuring various attributes of the windows such as hinting for the title, grabbing the mouse and keyboard, and accepting clipboard usage. In fact it may even be reasonable to complete some of this tonight.
Got the clipboard to work this evening, and a name displayed in the WM but that was all.

>>27815
How do you plan on implementing abort (C-g). My current approach is killing off all commands currently executing within a buffer by checking for the exit condition on wakeup and every potentially non-terminating loop condition.

>>27819
Liked the idea of using an interrupt to recycle a thread, handled in a wrapper. Maybe having ownership/batching for undo items and generalize the kill-ring to arbitrary storage so that pending changes can be automatically reverted. Would then need a special command to implement full abort (including undo) (C-u C-g). Programmers would need to be careful to reuse (or even free) the (now persistent) state. Really need to think about it more though, beyond the interrupt, this all sounds a bit too convoluted, and a bit like a functional memory-leak-atron.

File: 1734489137159.png (55.24 KB, 1920x1080, 2024-12-17-213138_1920x108….png)

Didn't get much software done today, but did get some other things done, and ended up feeling better. Made it where the window will appear with some scaled geometry in the middle of the screen. Also rediscovered TinyWM so will probably be using that as the basis for the window management component.

File: 1734563291978.png (27.54 KB, 816x703, 2024-12-18-180737_816x703_….png)

>>27805
>The simplest way to implement the latter can think of at the moment is to maintain a second cairo surface large enough to fit the full Pango layout and to copy rectangles from the large surface to the xlib surface. An optimization would be to perform these same functions but only render N lines at a time where N is computed from the window height divided by the line-spacing.
This is turning out to be difficult so figure would talk it through. Have the full_layout and compute pango_layout_xy_to_index() for the viewport origin. Substing to the line, then transpose the viewport_y minus the line offset, and to the viewport origin x. Next define operations to move the viewport origin as necessary. To support these means seperating, and intertwining the text representation of viewport, and some refactoring because of this. This is where was stuck. Pic related, is an attempt at an implementation, the tricky bit at the moment seems to be in how to compute the offset in the y-axis into the character, and specifically how to compute the descent of the character without reaching too far into the interface.

>>27796
Was nearly able to get away with using a single surface, and two layouts, one for the full buffer, and one for the visible lines. Made a demo of something like this but there is a seemingly unresolvable bug with the text overrunning or underrunning the window (because the surface is rendered twice). Also made a demo using two surfaces and this fixed the bug with text overrunning, but made the program quickly jump to the gigabytes in memory upon edits, and flicker with edits got beyond a sentence in length. Hopefully this is just a matter of not handling resources correctly.

File: 1734653462766.png (14.43 KB, 1920x1080, 2024-12-19-191042_1920x108….png)

Got the memory leak a bit under control, solving the CPU usage, and flickering issues. Am probably just going to leave the rest of the memory leak for now and work on other things until can rewrite in a garbage collected programming language. Not sure how many more features to add to this program before continuing either. Making a window manager is somehow a more tractable problem than making a simple text editor with internationaliziation and clipboard support. Doing so in a language with garbage collection might make it even more so.

>>27898
Actually managed to catch the memory leak now.

Have a less interesting bug which don't know how to solve: when the image surface that am copying from becomes greater than or equal to the surface am copying cairo doesn't seem to copy at all, and the screen is left blank. Any ideas on this would be appreciated. Other than this still haven't implemented verticle smooth scrolling, only horizontal because of a completely flawed approach to computing the offset into the character Y. Overall this project is turning out to be more difficult and time consuming than expected. bumped because slid.

>>27919
It seems the screen blanking only happens with certain characters, (it noticiably doesn't happen when the buffer ends in an 'M', or and 'H'). My best guess is that the program needs to do a better job of truncating the string of the layout for rendering.

>>27924
It actually seems to happen especially with the 'T' glyph in my font.

File: 1734812778415.png (19.54 KB, 1920x1080, 2024-12-21-152421_1920x108….png)

Couldn't figure out how to use ffmpeg to screen record so you'll have to just trust me and the screen shots but did in fact manage to get horizontal and vertical smooth scrolling. It currently only works with the trackpad or scrollwheel, and doubt its a complete solution, but conquered a set of small road blocks which is pleasant.

>>27929
Have mostly just been adding back functionality that was removed, or broken in recent iterations. Added back the cursor and the functionality to optimize by taking the horizontal slices, fixed a bug where the input buffer would delete everything to the end of the buffer if insertion or deletion wasn't at the end, and added the automatic movement of the viewport with the arrow keys, home, end, and backspace keys when at the edge of the viewport.

Would like to in the next few posts implement selection, and copying. This would put the implementation on par with a nano or pico editor. Of course to implement even a redumentary GNU Emacs is going to require implementing most of the subprograms.

>>27930
Should have included in this list text region highlighting, and opening/closing of files which are also required for implementing something like a pico or nano editor.

Please finally give us a good IDE

Have decided its best for my health to avoid imageboards entirely, even though this one is tame (probably because its a small technology board on a site dedicated to left-wing poliltics). Further developments on this project can be found at https://codeberg.org/jung/excerpts once the project has an initial commit. It will be licensed AGPLv3

>>27938
>Please finally give us a good IDE.
Will try my best.

Would be kino to have a hotkey that toggles between the classic IDE mode of viewing your code, files, and console output, and only having your code

>>28079
I'm almost sure you can already achieve this with both neovim and emacs

>>28079
>>28080 is right. In emacs you can save the current layout to register n with C-x r w n, make the current window take over the entire screen with C-x 1 and reload the previous layout with C-x r j n.

>>28080
>>28082
They're not IDE's though. Emacs could qualify as one but i fucking hate it. Jetbrains 4 life

>>28079
In Eclipse they are called "perspectives": https://help.eclipse.org/latest/index.jsp?topic=%2Forg.eclipse.platform.doc.user%2Fconcepts%2Fconcepts-4.htm

File: 1747835749787.png (362.8 KB, 1920x1080, 2025-05-21-093757_1920x108….png)

Instrumented this program (poorly) to figure out why it was slow. It turns out that it's almost entirely to do with pango_layout_set_text();

What you're supposed to do is incrementally render or at least update using pango_shape(); And getting the logical extents of each line for the purpose of computing the amount to render, and the same horizontal slicing technique. But instead this program is redoing the layout on any redraw of the screen.

It's also using the Xlib backend instead of the OpenGL one, this might also increase performance but not nearly as much as the above changes to the pangolayout rendering.

>>29884
The other major change being to rewrite it in Common Lisp of course.
Would have already helped some with doing proper instrumentation.

>>29884
This could also solve the issue with not being able to determine the correct size of the horizontal component of the viewport.
Or perhaps even the need to have a second buffer which is a slice of the other at all; because it's just the right size.
Although you'd probably need to create custom "empty" glyphs with the geometry set to the offset at each line (for when you've moved horizontally).

>>29886
You wouldn't add "empty" glyphs but rather render each line with an offset somehow because you'd always need partial or zero glyphs at the beginning.

File: 1747952927558.png (26.73 KB, 666x660, 2025-05-22-181248_1920x108….png)

>>29885
Started to rewrite with alien technology yesterday.

>>29884
>It's also using the Xlib backend instead of the OpenGL one.
cl-cairo2 doesn't have support for OpenGL so looks like we'll be sticking with Xlib. If the incremental updating etc. goes well it shouldn't really be necessary however on modern machines.

File: 1748028988390.png (35.69 KB, 708x773, 2025-05-23-152320_1920x108….png)

Just zombieing along, no real rhyme or reason currently. One thing runs out and another begins.

Got cl-cairo2 initialized. Not sure how to make the window easily accessible for the user to add XEvent maps.

File: 1748037140834.png (86.71 KB, 750x700, 2025-05-23-172116_1920x108….png)

Had a fun idea for a feature and algorithm. Basically text rendered to torus shaped surfaces and stuff like that.

It ends well, but am having some difficulty with start. The problems are the same in vertical Chinese of course.

>>29906
One idea is just use the same method but with search in the beginning.
Only problem is it might be too slow.

File: 1748136219993.png (58.31 KB, 1920x1080, 2025-05-24-205524_1920x108….png)

>>29907
Got basic text rendering today, just in a line. Handling of pango items but no way to easily specify attributes atm.

Arabic renders in a janky fashion without this and Mongolian apparently requires transformations to render correctly at all.

Would be very clearly better off writing this renderer in C and then using CFFI. The pango library doesn't really wrap SWIG much unless am missing something.

Assume it's not very painful to do this.

>>29920
>Mongolian
Taking this joke seriously is a bit dangerous…

Was told by my superiors am not ready for this.
And told paper and pencil would probably help.

File: 1748208195586-0.png (9.17 KB, 958x1080, 1.png)

File: 1748208195586-1.png (65.63 KB, 961x1080, 2.png)

>>29923
Some common lisp later and have a complex implementation of a simple algorithm that was in my notebook.
The idea was to only render glyphs which need to be on the screen, and only "shape" for glyphs that are in lines on the screen.
Arabic and other connected scripts are borked because of rendering glyph by glyph.
Seems need to separate by LogAttr is_word_boundary or similar ( https://docs.gtk.org/Pango/struct.LogAttr.html ).
Really this should have come up in the design process for the algorithm, but oh well.

File: 1748478353421-0.png (60.87 KB, 964x1080, 2.png)

File: 1748478353421-1.png (23.75 KB, 961x1078, 1.png)

>>29924
Writing in an alien language can be difficult!
Finally got all of the bugs sorted in the render!
The ASCII doesn't really show off the engine completely.
There's certainly some pressure to implement caching however.
Especially considering this was part of the whole reason for the re-implementation.
The algo avoids rendering words unless part of them is visible to the left.
There is still some work required to have this same behavior happen to the "right".
These things should be easily enough implemented now that the basic framework is in place.
Really still seem to need a non-idealist plan,

File: 1748570879179.png (13.7 KB, 960x1080, 2025-05-29-220404_1920x108….png)

>>30021
Wrote primitive caching. Also more primitive optimization for cluttered overlong lines.

It will probably be difficult to do this last thing correctly.

File: 1748626312805.png (194.17 KB, 1920x1080, 2025-05-30-133001_1920x108….png)

>>30039
Worked on handling a edge case today with vacant lines. My "solution" was not satisfactory.

It only tries to render things which are within the view-port.

Next thing on the list is to try to make a cached generator. The idea is that this would fix the problem with having to shape overlong lines. Might not be able to program this for a second.

File: 1748635094678-0.png (37.91 KB, 1200x278, foo.png)

File: 1748635094678-1.png (39.26 KB, 1200x294, foo1.png)

>>30045
Even with a hot cache only rendering what's necessary on a medium fast machine it's too slow.

More cold stuff.
Programmed and it turned out that a bulk of this was in marshaling and unmarshaling data.
Was able to half the consing, total-run-time, and nearly the real-time just by removing unused slots.
Pulling out the window attribute xcb calls (which never would have guessed) also took off 20 times real-time for small buffers.
And hot stuff.

The remaining problem is the lack of a cache generator.
Combined with running this for the full text instead of just the relevant view-port.
Together this meant it was taking way too long to shape up the characters for long documents.
Fixing this makes it too fast to create a flamegraph for even for large documents.
… but am sort of getting ahead of myself.

Evaluation took:
  0.396 seconds of real time
  0.336208 seconds of total run time (0.265805 user, 0.070403 system)
  84.85% CPU
  751,316,483 processor cycles
  8,931,360 bytes consed


Evaluation took:
  0.239 seconds of real time
  0.151740 seconds of total run time (0.094207 user, 0.057533 system)
  63.60% CPU
  452,475,538 processor cycles
  3,799,744 bytes consed


Evaluation took:
  0.010 seconds of real time
  0.021974 seconds of total run time (0.021974 user, 0.000000 system)
  220.00% CPU
  23,525,682 processor cycles
  694,528 bytes consed


Evaluation took:
  0.010 seconds of real time
  0.018074 seconds of total run time (0.018074 user, 0.000000 system)
  180.00% CPU
  15,426,824 processor cycles
  490,880 bytes consed


Unique IPs: 9
[Return][Go to top] [Catalog] | [Home][Post a Reply]
Delete Post [ ]
[ home / rules / faq ] [ overboard / sfw / alt ] [ leftypol / edu / labor / siberia / lgbt / latam / hobby / tech / games / anime / music / draw / AKM ] [ meta ] [ wiki / shop / tv / tiktok / twitter / patreon ] [ GET / ref / marx / booru ]
[ Return / Go to top /