The neverending quest to rewrite vichan -
Archived threads:
https://archive.is/xiA7y>>27025Yesterday gave the whole program a read through correcting errors along the way. Also expanded on the todos a little. Managed this morning to get thumbnail resizing setup to save on bandwidth. Would like to setup incremental autorefresh next, but don't exactly know how to do it without dropping messages, especially considering there are bulk operations.
Clerical
- host source
- public instance
Backend
- views for ban reports.
- easier way to create the admin account.
- LLM for moderation. (Llama Guard)
- prettier error pages.
- incremental autorefresh
>>27030Styled the error pages, cleaned up the error handling in source, and add proper handling for 500 errors in the mean time. The incremental refresh is turning out to be quite tricky. Was thinking could take a hash of the page to check for dropped messages, and just refresh the whole thing
like is currently always done in this case, but it turns out the preview script modifies the DOM and this is no longer a reliable measure. Am open to suggestions on how
of if its necessary to do this. Is the gevent loop just fast enough? The LLM has to wait for me to get a new computer to avoid swapping.
>>27030The only addition to the program after this morning was the simpler system to create admin accounts. Realized to setup incremental autorefresh would require reparsing posts to generate backlinks and at that point gave up for now. The view for reports of posts, users, and addresses requires me to make a query that don't know how to express in SQLAlchemy
think it would be an outer join, and coalesce. The LLM requires me to have a better computer to avoid swapping while developing. So that's it from me for now. Releasing the source is the only thing which seems possible for now, and still haven't decided on a platform for this.
Open to any suggestions.
>>27018I suggest you add the ability to run messages through an arbitrary server-side spam detection program. See
https://bogofilter.sourceforge.io for something like spamassassin, that also works for plaintext.
>>27045>ability to run messages through an arbitrary server-side spam detection program.Added, though this is a feature for advanced users given the implementation. Will use this for my LLM moderation system and have it as a separate program.
>>27045>beg for money.In the lupen sense and not the grifter, went ahead and removed it from the temporary nav.
Not a clue what boards to actually include either if it ever gets hosted.Also added stickies and locked stickies, styled the success pages, and cleaned up the error reporting some more.
>>27062Thank you! Avoiding the LLM is probably more practical because of the 2Gb of ram it eats up raising costs 50%. The idea was just to make moderation a little easier for those of us without thick skin.
>>27059Haven't yet found a way to write an ActivityPub server that would be reasonable. The FChannel implementation is many thousands of lines. The way the've implemented actors is also somewhat strange. Rather than having actors be a salt hashed address or a user account they've made them boards:
https://github.com/FChannel0/FChannel-Server/issues/9#issuecomment-822675142 There is a good JSON-LD library from the looks of it.
>>27134Don't suppose anyone here knows how to make SQLAlchemy return only the first image for each thread? So far as can tell am doing everything correctly here, but still getting an error that "row_number" doesn't exist. If it were just SQL could pull it off but for some reason the "label" even as it emits an AS does not cooperate with the following use of the labeled item.
Unrelated am also having an issue with Starlette-Login trying to serialize user to JSON for some reason and failing at this. Should be able to take a deeper look into this myself at some point.
>>27168Is there someway to have two hashes for the same object and then confirm that they are for the same object without knowing what that object is? Was thinking for moderation purposes it would be interesting to have hashes of addresses communicated between federated servers along with their respective messages in such a way that each could check whether or not its an already banned user. Found this:
https://crypto.stackexchange.com/a/102963 but it seems a little theoretical, so probably not.
>>27170>Hopefully sometime tomorrow can finish off the migrations with its comprehensive testing,Wouldn't say the testing was quite comprehensive, but did a quick read through, and some sporadic testing which turned out well. Went ahead and merged the Starlette branch with main.
>>27170>begin to implement new features again.Am considering getting rid of the (you)s to enable caching to work more effectively. Theoretically post identifiers could be stored in the session, and this way the (you)s added back at a later date. Am also considering replacing the server side rendering of the [logout] button with client side rendering. The downside is slightly less functionality for noscript users, the advantage is that caching is much easier.
>>27174Made the image and reply count appear on the catalog page. Also added the user and address reports to the reported page. Been working through bugs otherwise. Here are my remaining TODOs:
Devops- host source
- public instance
Missing- RSS feeds.
- ActivityPub.
- post search?
Most of what the program needs at the moment is testing, to uncover bugs and then subsequently fix them. Have been doing some of this already today, but it may be about time to finally get the source code hosted somewhere and start receiving feedback. Biggest hold up here is deciding on the platform; seems like it's GitHub or Codeberg.
>>27178Don't have it in me to write documentation at the moment but here's the repository:
https://codeberg.org/jung/arsvia Plan to add install instructions, and some pictures. The gist is:
1. install postgres, imagemagick (libmagickwand-dev), python3, and pip.
2. run pip install -e .
3. setup a user and database in postgress and add to config.py
4. python3 arsvia
5. navigate to 127.0.0.1:5000
6. create an account which will automatically become an admin.
>>28343Make everyone register an account
(for real tho this could be not bad. It would help against spam and illegal content by raising the barrier to entry some [and new account creation could be temporarily halted, which is better than harming vpn or tor users or etc], and on the front everyone could still be anonymous. Or a tripfag at their choosing. whatever. Would it be so bad?)
>>27187Am considering rewriting arsvia, my imageboard, in typescript (using Next.js and TypeORM) to expand my techstack.
Arsvia has user pages containing all their posts, and thought to run with that idea a little.
Namely adding a feature for sharing usernames either with specific people or making them public.
And more interestingly, to allow users to assign arbitrary nicknames to other users, and to post to user pages without a board.
Guess the user pages could reuse the moderation functionality so that users could manage their own pages.
Maybe with the Oath2 JWT authentication and try to figure out email verification too.
In short it would be something in between an imageboard and a microblogging platform.
>>30504Am thinking for this version instead of using the UUIDs directly I'm going to rewrite in the render such that it generates a sequential ordering by creation time followed by internal UUID. For threads there could then be an array perhaps with ranges or an epoch to avoid the ordering reusing numbers for the render (pretty ugly imo).
Further I'm going to use threads as namespaces like in textboard.org so that each thread starts from zero.
I'm also considering using the UUID primary key of the address log table to emulate accounts for anonymous users. This would mean that they could be federated if that was something of interest.
Could consider also using readable UUIDs like
https://github.com/Debdut/uuid-readable though this is a little chaotic, but at least they wouldn't be like Ganymedean's from PKD's "The World Jones Made".
>>30505Interesting. While working on the successor to
>>>/siberia/679700 i settled on using a tai64n timestamp with 2 extra bytes of entropy (prepended to the start, so posts and threads can be distributed into buckets). This also means my thread files, which consist of lines of these IDs, have fixed-size records and can be atomatically added to using O_APPEND.
>>30510Downloaded, and read the source for the three main SSG programs, and skimmed io.c.
It's obviously a very interesting approach.
Just from reading what you wrote here for some reason thought had made the program to use fixed disk space.
That it's a textboard has reminded me that I've no idea how to federate files.
>>30514Thank you. If i had to summarize this approach, it would be reducing complexity by finding the right interfaces.
>That it's a textboard has reminded me that I've no idea how to federate files.Elaborate.
>>30515>right interfacesWouldn't have guessed that this is what you would have called this.
Don't worry about the files thing was just being foolish.
Should probably spend some time learning UNIX a little better.
If for no other reason than to get better at using other programming languages' standard libraries.
>>30514In other news seem to have gotten something like a nearly complete schema setup:
auth
├── PrivateAddress.ts
├── PrivateUser.ts
├── PublicAddress.ts
├── PublicUser.ts
└── UserRole.ts
reports
├── PostAction.ts
├── PostReport.ts
├── Report.ts
├── UserAction.ts
└── UserReport.ts
threads
├── File.ts
├── Nicknames.ts
├── PageInfo.ts
├── Post.ts
├── Reference.ts
├── Tags.ts
└── UserProfile.ts
3 directories, 17 files
The most interesting features are the ability to assign nicknames to anonymized (by default) IP addresses or users.
And the replacement of boards with moderated user created tags; along with the posting to multiple tags.
Am thinking of federating anonymous posts using throwaway
https://github.com/Debdut/uuid-readable UUID usernames.
This would be as an alternative to the high dox potential of having anonymous histories based on IP.
>>30516>to get better at using other programming languages' standard libraries.C doesn't really have the best standard library. You usually want to avoid using anything that isn't a thin wrapper around a syscall.
>Wouldn't have guessed that this is what you would have called this.I got this from
https://skarnet.org/software/skalibs/djblegacy.html<One of the "DJB philosophy" key points is to question the interfaces. You have a task to do; you have existing interfaces. What do you do?<Interfaces should be questioned right down to the libc. You cannot build strong software on flakey foundations. And from a system and network programmer's point of view, one thing is clear: most standard libc interfaces suck. There is no buffered asynchronous I/O. There is no timed I/O. There is no heap management helper. Even simple system calls are not always guaranteed to succeed!UNIX doesn't have proper records, which most developers compensate for with databases, essentially a second filesystem layer. Files and directories are really the only level where database operations are simple, reliable and fast, so if you keep your state simple enough, you can map all of it onto simple file creation and access.
The next version will allow composing posts by processing arbitrary blobs, but i'm proud of how the static site generation approach worked out for this one. The locks are the only non-essential state of the system and usually short-lived, everything else is either persistent data, static configuration or entirely ephemeral.
>>30516Am probably going to restart.
I've realized this should be a ActivityPub server first.
And only secondly a anonymous imageboard (front-end).
Unique IPs: 6