/tech/ - Technology

>>27738
Have been thinking some about making an Scheme compiler based on the one provided in PAIP (see the OP picture), but this has very limited real world applicability. Am really looking for something which is both marketable and a reasonable technical challenge. >>27553 seems to have been insufficient on the latter front, and even though >>27187 was good fun the lessons have now been learned. Other items on my list are an Emacs, and a minimal web browser. Unfortunately don't beleive either of these is marketable either.

>>27738
a "deferred mode" gui toolkit. usually gui toolkits are divided in two categories: immediate mode and retained mode. im is good for games I guess but if you want to write a gui for regular software they both suck, im is kind of incompatible with features one would expect from a gui. rm is bloated by design, the concept requires taking over the main event loop, this is why so many rm libraries end up creeping into unrelated areas (for example, both qt and gtk force you to use their own api for threads, sockets, polling, io, etc.)

I wanted to write a gui library that would describe the gui as a tree, kind of like the "document object model" that browsers use to represent parsed html. this tree could be created either by parsing some mark format (like html) or programmatically. the gui state (the node currently on focus, selections, event handlers, user data, etc.) could be stored either in that same tree or in a parallel state tree. events would be deferred through an api similar to the xcb xcb_get_file_descriptor and xcb_poll_for_event method. the point is that the gui wouldn't block or take over the main loop, so you could use your own polling strategies and event loops

but I don't have the time to actually write it. are my ramblings too cryptic? for the user, the api would look somewhat like this (pseudo-code):

fn main() {
    UI.init();
    var tree = UI.Tree.fromFile("gui.ml");
    var state = UI.State.fromTree(tree);
    var fd = UI.getFd();
    while(poll(fd)) {
        const event = UI.getEvent();
        UI.update(&tree, &state, event);
        UI.render(tree, state);
    }
    UI.deinit();
}

as long as it was made to use the c abi it could be ported to basically any language

>>27739
>Unfortunately don't beleive either of these is marketable either.
Let's not deceive ourselves, currently the only ways to make money from software is to heavily capitalize on a strong existing, be with an established corpo or build a community around the software you can siphon donations from. As the latter seems to be the only viable option to you, a novel text editor or web browser would definitely attract some attention.
>>27741
Tried that in CL once and it yielded an overgeneralized pile of nothing. Designing interfaces is hard.

I plan to write an emacs based on a forth with only strings and object datatypes. The core editor would only manage and display buffers, commands would be passed over rpc to multiple backends e.g. a file server or www gateway, most-likely as part of some kind of super-server. The protocol is designed around batching large strings, but i'm not far enough into development to measure anything. If emacsclient can get away with communicating over TCP, so should an editor backend.

>>27742
I have a working example in c that uses html for the mark language, but that I don't have the time to work on it anymore. I don't think you understand the concept I was describing, and I don't understand how do you even produced your "overgeneralized pile of nothing", it is a pretty straightforward idea - much simpler than any retained mode gui if you have ever wrote one. is this bait?

>>27745
>I have a working example in c that uses html for the mark language
You should probably stick with HTML, XML or something similar. I've gone through 2 or 3 revisions with my own DOM. The code still loooks salvageable, but as i said it has become far to generalized. Good cross-platform UIs are always limited in some and dictate their own mode of interaction. There will never be a fully-featured DOM that looks equally intelligible on X and a VT100. This is the comment on top of the file:

;;;; high-level description of user interaction
;;;; a ui-tree is constructed from nodes and leafs,
;;;; described with define-ui-node and defui
;;;; the leafs are ui-elements that are usually instantiated,
;;;; when a bind-element-to-ui call creates them under their assigned node
;;;; nodes pass control flow to any of their children
;;;; based on the elements present on a stack,
;;;; when reaching a leaf it calls a function based on the stack element
;;;; ui-tree interaction emulates a 0-address, single-register stack machine,
;;;; with the supplied data acting as an opcode
;;;;
;;;; tree := element
;;;; element := ui-element | node
;;;; node := ({predicate element}+)
;;;; ui-element := ({(type action)}+)
;;;; action := (λ () program)

>I don't think you understand the concept I was describing
While i can see why a DOM would be a good way to get around some limitations of a rendering pipeline, i dont't understand why you're reluctant to let the graphics framework take over the event loop. Most types of applications could be wrapped in a single callback. If the target platform has working IPC, fork() could also be used by whatever graphics init function.

Controlling the event loop isn't even useful most of the time. You always either want a fixed framerate or updates as needed. Most events should be processed or discarded immediatly, so again callbacks or a worker process are optimal.

>>27752
>Controlling the event loop isn't even useful most of the time.
maybe it isn't useful for you
suppose you are writing a gui program for torrents. you have to use sockets, many of them at the same time, some as server and some as client, not all of them using the same protocol. you probably also want to use a libaio style event loop for reading files and doing integrity checks. and while doing all of this you want the program to stay responsive to user input. all of these things will alter with the interface tree (to move a progress bar, to show a download/upload speed, etc. etc.) so lets see the options if you concede the event loop solely to the gui:

a. using threads for everything else. in this case the gui event loop will have to use mutexes to make the tree and state thread-safe. the threads will have to block (killing performance) or poll and defer based on mutex availability, which would significantly increase the complexity and still hurt performance somewhat

b. adding an api to the gui loop to support all the mentioned io configurations, and then do the same with every possible use case. this is what most retained mode libraries do, it hurts modularity, separation of concerns, and code reusability. it adds a significant work load for both the programmer, who now has to cover all the possible use cases in a complex area that has little to do with graphical interfaces, and the user, that has to learn and adapt to the particularities of these ad-hoc wrappers. this also increases the complexity of things like error handling and partial deinitialization on error because you are doing everything inside callbacks. another thing it makes more complex, almost impractical, is testing, because it becomes harder to cleanly recreate test scenarios

I don't think you understand how ipc works. if you have a child process it still has to communicate with the parent, either through a pipe, socket, file, or some other interface. so you are back at square 1, and if anything, you have added some complexity to the event loop now (checking the child process status, etc.)

think about it this way, the gui event loop is by far the simplest one: it blocks on a single fd. there is no reason why this should appropriate control flow when it could be integrated into a more generalized form (polling a set of file descriptors, for example). you can still have fixed framerates and/or event-based rendering with an user event loop + you don't lose orthogonality, separation of concerns, performance, etc. im libraries already do this and they are easier to use than rm libraries, and easier to integrate into different scenarios. not to mention that the code becomes easier to read and overall better

tl;dr if you want to see what I mean just write a small telnet gui, first with gtk using the glib event loop, and then with xcb or xlib using your own event loop (the poll syscall, or threads + a thread safe event queue, or whatever else you choose)

>>27754
>using threads for everything else. in this case the gui event loop will have to use mutexes to make the tree and state thread-safe.
Some workloads can be synchronized with atomic data structures, like a ringbuffer with a single reader and writer. Generic thread-safe code has many synchronization pessimisms, that a tight coupling between graphics api and renderer can avoid.
>if you want to see what I mean just write a small telnet gui, first with gtk using the glib event loop, and then with xcb or xlib using your own event loop
I wrote a wrapper around SDL2 for a small project ( >>>/games/38378 ) and arrived at a program/library split similar to retained mode graphics. The library handles all texture allocation and to avoid reissuing commands on every redisplay, i had to reimplement SDL2's internal batching of drawing operations. Therefore, despite at the moment relying on the unspecified behavior of rendering inside callbacks, i'm very close to shunting SDL_Init into its own thread.
>I don't think you understand how ipc works.
With the same kind of data structures as found in multithreaded code you would obviously only have the added complexity of memory sharing, but pipes are more ideal for some data. Unix signals also open up the possibility of interrupting the other process.

You also seem to be under the impression locks and blocking io are bad in every instance. Blocking on user input usually doesn't noticeably affect performance and it makes more than up for it in simplicity.

If i had to offer a universal solution to your conundrum, i would suggest structuring the application one or more queues and handling all ui interaction via callbacks. That way your backends would have it's own event loop with blocking or polling library calls and the logic for targeting specific framerates or redisplaying on focus would live entirely in the library.
Flood detected; Post discarded.

Currently trying to learn go by making a web service that cuts your text into tweet-size chunks
>>27754
You may want to check out the Sans-IO pattern https://sans-io.readthedocs.io/

>>27758
>graphics api and renderer
not what I was talking about

>a ringbuffer
if you defer events you are back at the problem of polling. this is, the main loop of the gui is waiting on an x11 connection, so besides the buffer you also need either a self pipe or an eventfd syscall to have something you can poll besides the x11 connection and the preemption timer. at this point, you might as well generalize the event queue itself into what I described here:
<events would be deferred through an api similar to the xcb xcb_get_file_descriptor and xcb_poll_for_event method.
and then the main loop becomes so simple you can leave it open for the user, which will also make subsequent code more simple. think of how sdl exposes SDL_PollEvent and SDL_PushEvent to the user, and how im libraries then take that input without taking over the main loop

>With the same kind of data structures
that's not the point. the point of the idea is having better event loops, ipc doesn't change the fundamental problem - it is just a fd and the added complexity of keeping track of children processes

>You also seem to be under the impression
read my posts more carefully then. this has nothing to do with blocking, on the contrary, the point is to block. to block, you need your events to be (1) uniform and (2) supported by the os. for example, on linux you can block on file descriptors and you can block on mutexes - but you can't block on both. you can poll on a set of fds, but not on a set of fds and mutexes. blocking is already a delicate matter, when a library adds abstractions on top of it, it can't really change the essence of the situation, it just adds complexity and reduces flexibility

you seem to think I'm talking about fancy multithreading rendering or something like that. I'm not. if you could try writing even a basic telnet gui you would probably understand what I mean. I'm talking about how inadequate the retained mode is when you have to, say, poll on 200 file descriptors besides user input. the big libraries can do it, they have a pretty big api to wrap syscalls like epoll, but it is clearly the wrong approach. the same thing could be expressed much better and with more space for optimizations just by leaving the main loop to the user

>ui interaction via callbacks.
<the gui state (the node currently on focus, selections, event handlers, user data, etc.) could be stored either in that same tree or in a parallel state tree.
again, if you expose the event queue, might as well go all the way and expose the loop, which, if the library is well designed, should look like >>>/tech/27741

>>27765
>you seem to think I'm talking about fancy multithreading rendering
I'm the one talking about it, or not at all actually. For a VT100 application you can get away with only writing to stdout once when flushing the screen state and during redisplay in a few signal handlers. If you want the same kind of interface simplicity for GPU rendering, you can't have the renderer take over the applications event loop.

What i'm trying to say is, the structure of the display event loop itself should be an implementation detail, except for the fact it doesn't interfere with the applications event loop. The application should transparently put graphics on the screen without implementing things like redisplay on window focus and be able to block/poll on any user events or other sources.

>I'm talking about how inadequate the retained mode
I wrote Hello World in gtk+ many years ago. It's all coming back to me now. So what it needs the programmer to do is invoke the gtk event loop as part of the main event loop and let it do its internal polling. This is objectively bad for applications doing their own asynchronous io source and adding additional input source functionality to the toolkit seems like an overly complex kludge.

On the other hand, the approach was likely chosen so applications could run on a wide variety of displays without multithreading as an explicit requirement. With multithreading you could run a gtk instance in its own thread, even though you would need to duplicate existing functionality. Would you conceptually be happy with something like gtk, if it had a non-blocking Gtk_Main?
Flood detected; Post discarded.

Mario type of game in C/C++ that uses minifb and the least amount of libraries possible, to learn low level multimedia programming
https://github.com/emoon/minifb

Don't suppose anyone has a good example program using Pango, Cairo, and Xlib lieing around? Seem to have spent the better part of the afternoon trying to glue together a functioning XLib + cairo display tutorial and a functioning pango + cairo tutorial to no avail. The documentation is lacking to say the least.

>>27770
Found at least one decent one and was able to make a little progress now their is just that the Expose events don't seem to be firing. Will probably make a new slower thread soon concerning this new project, and also try to understand some of the conversation that's happened here.

>>27773
>now their is just that the Expose events don't seem to be firing
This is also solved, just needed to set the ExposureMask.

>>27767
>On the other hand, the approach was likely chosen so applications could run on a wide variety of displays without multithreading as an explicit requirement.
except you don't need to do that to support many displays, and it has nothing to do with threads

>Would you conceptually be happy with something like gtk, if it had a non-blocking Gtk_Main?
no, and gtk has bigger problems. maybe a non-appropriative qt, but rewriting these libraries in a different style would miss the opportunities for improvement. a better approximation using existing libraries would be something like nuklear but with a more declarative api and better features (through some "retained" state exposed to the user) like the "selectable region" from flutter, which is easy to implement when the ui is structured as a dom-like tree, but is hard in the immediate mode where there is basically no structure

Another idea:
I assume everyone here knows about event loops by now, right? the idea is that you use a kernel feature like epoll or iocp so that your thread can wait on different blocking operations simultaneously. blocking primitives are usually simple, so you can describe them with a simple data structure, and then use a queue with that structure.

so they usually work like this; instead of calling blocking operations directly, you just add a "task request" to the loop queue, for example {OP_READ, my_file, &buffer, callback, &user_data}. then you run the loop, which will take tasks from the queue and block on the entire set using the backend (epoll, iocp, etc.) api. once one of the tasks becomes ready, the pool performs the task (reading from the file into the buffer, in our example) and calls the provided callback with the results of the task if appropriate (the buffer) and the provided pointer to user data, a common patter in c

because of this, code written to work with event loops like libuv, have to be structured as chains of callbacks. the entire state for each chain of callbacks (for example, to handle a client connection to a server) is passed using that user_data pointer. it is efficient but hard to read, hard to debug, and error prone. maybe if processes, threads and context switches weren't so expensive, all of this could be avoided in favor of either child processes or threads, which have a cleaner interface. but that's simply not the case

so I have been thinking, interpreted languages usually keep their entire state in a single vm or context object, so what if instead of callbacks in the compiled language, you exposed to an embedded interpreted language an api which would, under the hood, add a task where the ud points to that context. the callback would simply push the task result as the return from that api, and resume the execution of the interpreted script from there. this way the user could write async programs without the callback hell

>>28056
>maybe if processes, threads and context switches weren't so expensive, all of this could be avoided in favor of either child processes or threads
http://kegel.com/c10k.html#threaded and the linked paper make a similar argument. then again even though threads and processes can be expensive, the overhead only matters with large-scale workloads. after all forking on each client connection is still the standard c/unix model.
>so what if instead of callbacks in the compiled language, you exposed to an embedded interpreted language an api
https://en.wikipedia.org/wiki/green_thread?useskin=vector

>>28057
yeah, it would be like a process-level non-preemptive scheduler like green threads. another thing is that you could use this same gimmick with an actual thread-pool. as in, you could have a thread per cpu core, and run an event loop on each thread. because the state of the interpreted language can be passed around, the user program could be moved between different threads

what I mean is that the provided api would add the task to a common queue, and then some worker thread would consume from the common queue and add the request to it's event loop. backends like epoll usually support a timeout parameter, which could be used for a load balancing algorithm. an event loop with many tasks spends less time blocking; we can use a timer event to check if the el is spending too much time blocking and try to consume new requests from the common queue

all of this would be transparent to the user of the interpreted language, but it would maximize throughput even under very heavy loads