Every few months, people come up with an idea to create a new imageboard, since mostly everyone is dissatisfied with the state of vichan/lainchan, or thinks they can do it better. Having better software would greatly enhance the experience for both regular users and mods, and have more crossover appeal to “normies”. The problem is that no one can agree on the technical or more importantly non technical decisions on how one would go about making an actual, usable replacement for lainchan.
In fact, people don't even agree on whether the imageboard replacement should be an imageboard at all. This thread is a merged, consolidated, megathread of all the various attempts at answering this question that people have made.
Previously, there was a thread on outreach to lainchan, including a strawpoll:
https://lainchan.org/%CE%BB/res/26674.htmlThe poll determined it should be built in Java, but a significant minority wanted to use a functional programming language, esp. Haskell, or Clojure/Lisp.
The only way a new imageboard will be built is if multiple people, technical jannies and lurker-programmers, from here and lainchan and even elsewhere, actually collaborate on a single project and concentrate their efforts on this.
There are important technical and non technical questions to be answered.
Namely, do we even want an imageboard?
Some anons have suggested a radical re-imagining of the medium of anonymous communication where boards are replaced with tags. Some want named accounts, some want textboards, the ideas are endless. Is there anything to be improved upon with the imageboard concept, and if so, what? If reddit is an improved version of the phpbb type forums of old, what does a “modern” version of a chan look like?
Feel free to use this thread for any polls, discussion, of the formats, technical, or non technical decisions related to a vichan replacement both for this site and in general.
Also comment on:
Concept: Classic Imageboard vs something else
Architecture: Monolith vs Microservices
Distributed: Distributed/Federated vs Standalone
Front end: SPA (ex: Angular, React) vs Server Side HTML templating
Backend Language: Java, C#, Lisp, Rust, Golang, PHP, etc.
Database: SQL vs NoSQL
How close is GOCHAN to being done? we need to replace this shit lynxchan software soon.
there's also 'jschan'
https://zchan.cc/index.htmlI will just copy paste what i wrote in another thread
Currently the rough roadmap to minimum feature completeness is this:
>1. Refactor the entire database design to be magnitudes better than before, follow relational database standards in terms of normalision, and support planned future features. Implement sql queries to glue all the old functionality to the new database design and redesign that code to follow good code practices. Unify the 3 seperate sql build scripts into one master file.
That is basically done
>2. Write a migration script to port old gochan databases to the new format. Implement a versioning system for future database revisions that will work in a similar fashion to entity framework migrations (but manually written). Test the whole thing thoroughly.
This is where we are right now. After that
>3. Create a more specific roadmap with small individual tasks that people can pick up and perform. Which includes:
>4. Overhauling the banning system which is super different from the old one, to use the new functionality and logic. Remove old banning system
>5. Allow multiple files to be uploaded per post (database already supports this)
>6. Implement a variety of frontend usability such as autorefreshing, expanding images, etc
>7. Make the frontend match closer to the bunkerchan lynxchan frontend, including porting themes
>8. A testrun by the mods and users
>9. Writing code to migrate the lynxchan mongodb data to the new sql database.
And that is roughly our roadmap. Keep in mind that the new db design and refactoring the entire codebase to use it (especially since I, the person doing the refactor, was unfamiliar with the codebase) is the most difficult part, and also the part that prevents others from working on other issues simultaneously due to the code and database changing massively constantly.
—–
I am currently trying to write sql to port the old db to the new schema, but having to support two versions of sql makes it fucky and also the fact that the old database was completely unnormalized so I have to split certain tables into three new ones which all refer to each other with foreign keys.
Drop by in the leftypol riot and @me if you wanna help out, though we do plan on making an official roadmap/todo board for people here to help out with small well defined tasks once the database thing is over.
Please be patient. I do value the work I put in a lot but despite having graduated as a programmer recently, programming for prolonged times makes me anxious, mentally numb, unfillfilled and tired.
unimportant but maybe contextfull: We just officially dropped sqlite support because it doesnt support table alterations after a lot of trouble. I am also trying to split a self-referential table containing all posts + any accompanying files/images into a thread, posts, and files table, all 1 to many, to support multiple files and also to more cleanly seperate thread entities from post entities.
>>1952>write an addon with my 1337 addon system (addon system is just runtime monkey patching and as such plugins may and do break every update, there is no actual addon system)It gives you the most freedom, but I agree that an actual plugin system would probably be better.
>code is an undocumented messThere are about 6k non-empty lines in doc/*.txt. The comments inside the code base don't explain anything, true. Though most of the time it's easy to understand what's going on.
>the unwillingness of stephenlynx to accept pull requests>consider features we are unable to implement features the mods of this website would likeWhen we migrated from Vichan to LynxChan last year, StephenLynx implemented a lot of features for us in an addon which he later added to the engine (post history, DNSBL, ASN bans, individual file actions and more). Yeah, it's not very easy to get your suggestions or PRs accepted if you don't discuss them at length with him on IRC or on /lynx/.
>link to sitehttps://kohlchan.net>We have 145k unique users each monthWe have twice as many unique users each month according to Cloudflare Analytics. It's not a very reliable metric since many people have dynamic IPs which change every day. I'm only making the point here that LynxChan easily scales and our server is not even very powerful.
>>1955
>Yeah, it's not very easy to get your suggestions or PRs accepted if you don't discuss them at length with him on IRC or on /lynx/.
He flat out told us he would not accept any pull requests, ever. He also kept being extremely smug and condescending.
>>1955
>It gives you the most freedom
Yeah but at this point you might as well just directly modify the code.
Now that is not to say gochan is currently at the level of lynxchan (golang has no stable plugin/monkeypatching support), but for us, considering the future direction of features the mods want to go in, we would need such substantial changes to the codebase we might as well have created a different branch. On top of that, space and I would rather use an sql database.
So far the code layout and inner architecture of gochan is more comprehensible than lynxchan was to me. Im not going to list all of the things that are better, since it would just seem like im throwing a hissyfit.
But gochan was already in the books a year ago, it gives us a lot more control over it, and we think that by being a lot more open about collaboration (and not, say, tell people we wont accept pull requests ever) we can more quickly realise new features we need. If only for our own site.
Though I appreciate any info you have on what could cause issues for the performance of lynxchan beyond being an old version. Dankse sehr.
>>1956
space still hasnt given me access to the server so i cannot check them either
>>1959So it's still caching to disk, has functions with over 300 lines (
https://github.com/gochan-org/gochan/blob/master/src/posting.go#L276 ), no lazy/JIT caching and zero documentation? On a second thought, I can't find these source files in this branch. Did I miss anything or is this some sort of rewrite?
>>1960Nvm, found it.
It's almost 400 LOC now.
https://github.com/gochan-org/gochan/blob/Database_refactor/pkg/posting/post.go#L37>So far the code layout and inner architecture of gochan is more comprehensible than lynxchan was to me.🤔
>>1965
>I dont go into chats and proceed to make fun of other peoples personal life as an opening statement.
>The what now?
Dont pretend i didnt see you trying to make fun of me saying "i dont like programming for hours on end" in the leftypol chat on riot.
>You don't even go to fucking #bunkerchan anymore.
Exactly. Not sure why you even come here, to this site, on this board, in this thread, just to try and talk shit. Youre not even socialist, according to yourself.
>Then why aren't you addressing it?
Because im not going to address your childish attempt at talking smack. Especially since I did not write that code you are referring to in the first place. It should be obvious that software still in development, hence why we havent switched to it yet, is not feature complete yet.
That function should be shorter. It should have better caching. But right now my priority is finishing the database, adding multifile support and several other things, not cutting up a working function, that is going to get reworked soonish anyway, into smaller parts. And unlike with lynxchan, I and basically anyone else who wants to help can go ahead and fix things like those, and not run into "i dont accept any pull requests".
>If you are so sensitive to pixels on the internet, don't look at them.
You said the same thing about child porn but unlike sociopaths like yourself, most people do have basic human emotions.
>>1984
Sure, but can you extend gochan AT ALL? Like, without a fork? Lynxchan might not offer you a tidy api, but you do have flexibility. Monkeypatching is just changing attributes on things, it might be shitty when you change stuff like how your strings behave, but we are talking about the highest level logic here.
As for "confusing", i'd say that's not only subjective but also I have been praised more often than not for it's organization.
And mongodb is fine. More than fine, using mongodb gives you a much more practical workflow where you don't have to update a schema for every single little change you make to the model. Not to mention how the management tools are more modern and efficient. For example: for you to allow a remote connection on myslq you have to properly register it directly on the database along with the user. With mongo you just do the more intuitive thing of telling it to bind on 0.0.0.0 and connect as you would usually do. Now, to be fair, gridfs did have a huge drawback I didn't consider at the time: memory usage. You have a shitload of documents for the binary data and they have an index around the file they belong to. Which is no longer mandatory on 2.4, so that's not really an issue now.
I've come to make an announcement: Stephen Lynx is a bitch-ass motherfucker, who pissed spaghetti code on my image board. That's right, he took his tiny fucking dick out and he pissed on my fucking imageboard, and he said his software was "THIS GOOD", and I said that's stupid, so I'm making a callout post on my riot IRC, Stephen Lynx, you got a small dick, it's the size of this walnut except WAY smaller. And guess what? Here's what a good image board looks like! THAT'S RIGHT, BABY. FUNCTIONAL TOOLS, NO CRASHES, NO SPAM, LOOK AT THAT, IT’S MADE IN FUCKING GO. He fucked my imageboard, so guess what? I'M GONNA FUCK AMERICA! That's right, this is what you get! MY SUPER LASER PISS CODE! Except I'm not gonna piss on America. I'm gonna go higher. I'M PISSING ON CAPITALISM! How do you like that, OBAMA? I PISSED ON THE FLAG, YOU IDIOT! You have TWENTY THREE HOURS before the piss DRRRRRROPLETS hit your fucking github, now get out of my fucking sight before I piss on you too!
>>6625Javascript land is insanely insecure and a fucking mess. As long as you keep your dependency graph relatively small and shallow, you should be fine not updating your libraries in a while.
There are also other solutions, like using a language that compiles to JS. If on top of that you use a language that is strongly typed, you probably won't have many security problems and will probably be fine staying with whatever version you have. And if you ever update, the compiler can help you out.
There are downsides to that as well, it's not a silver bullet.
I'm actually very slowly writing an API for an imageboard. I'm using it to learn, mostly. If I do it right, I guess I'll publish it. To get feature parity is pretty hard. For example, post.php of lainchan is 1530 lines of code. It's a fucking mess and can probably be written in half of those lines, but that is just one of many many files.
To give an idea, this is lainchan:
17 403.php
34 404.php
15 banned.php
15 banners.php
20 bg.php
20 c.php
1006 install.php
24 log.php
216 mod.php
28 player.php
1530 post.php
19 report.php
174 search.php
86 smart_build.php
54 staffapplication.php
298 inc/anti-bot.php
209 inc/api.php
320 inc/bans.php
173 inc/cache.php
1897 inc/config.php
108 inc/controller.php
165 inc/database.php
451 inc/display.php
91 inc/error.php
45 inc/events.php
253 inc/filters.php
2889 inc/functions.php
502 inc/image.php
366 inc/instance-config.php
39 inc/lock.php
187 inc/polyfill.php
49 inc/queue.php
64 inc/remote.php
65 inc/route.php
78 inc/template.php
53 templates/themes/basic/info.php
40 templates/themes/basic/theme.php
57 templates/themes/calendar/calendarpost.php
36 templates/themes/calendar/info.php
40 templates/themes/calendar/theme.php
110 templates/themes/catalog/info.php
506 templates/themes/catalog/theme.php
42 templates/themes/categories-uboachan/info.php
223 templates/themes/categories-uboachan/theme.php
67 templates/themes/categories/info.php
88 templates/themes/categories/theme.php
81 templates/themes/donate/info.php
28 templates/themes/donate/theme.php
36 templates/themes/faq/info.php
33 templates/themes/faq/theme.php
55 templates/themes/frameset/info.php
61 templates/themes/frameset/theme.php
38 templates/themes/irc/info.php
27 templates/themes/irc/theme.php
33 templates/themes/public_banlist/info.php
57 templates/themes/public_banlist/theme.php
58 templates/themes/radio/info.php
27 templates/themes/radio/theme.php
54 templates/themes/rand/info.php
105 templates/themes/rand/theme.php
88 templates/themes/recent/info.php
167 templates/themes/recent/theme.php
75 templates/themes/rss/info.php
198 templates/themes/rss/theme.php
22 templates/themes/rules/info.php
27 templates/themes/rules/theme.php
70 templates/themes/semirand/info.php
257 templates/themes/semirand/theme.php
62 templates/themes/sitemap/info.php
48 templates/themes/sitemap/theme.php
36 templates/themes/staffapplication/info.php
54 templates/themes/staffapplication/staffapplicationpost.php
40 templates/themes/staffapplication/theme.php
99 templates/themes/stream/info.php
34 templates/themes/stream/theme.php
23 templates/themes/zine/info.php
28 templates/themes/zine/theme.php
14790 total
That's 14k lines of code. A lot of these are not used at all or are config shit. The codebase has a huge issue of dead code.
Perhaps a more "fair" assessment might be this:
17 403.php
34 404.php
15 banned.php
15 banners.php
20 bg.php
20 c.php
1006 install.php
24 log.php
216 mod.php
28 player.php
1530 post.php
19 report.php
174 search.php
86 smart_build.php
54 staffapplication.php
298 inc/anti-bot.php
209 inc/api.php
320 inc/bans.php
173 inc/cache.php
108 inc/controller.php
165 inc/database.php
91 inc/error.php
45 inc/events.php
253 inc/filters.php
2889 inc/functions.php
502 inc/image.php
366 inc/instance-config.php
39 inc/lock.php
187 inc/polyfill.php
49 inc/queue.php
64 inc/remote.php
65 inc/route.php
78 inc/template.php
9159 total
Say you can write this if you have experience and acumen in 4.5k lines of code. I've heard that people write 20 lines of code a day on shitty desk jobs (
https://skeptics.stackexchange.com/questions/17224/do-professional-software-developers-write-an-average-of-10-lines-of-code-per-day). If that's true, this would take a full year of work. Something more reasonable, since you already know what your building and this is more personal work is to write maybe 40 lines of code on average during the whole endeavor. That's around half a year of dev hours. These things are really hard to estimate, and they're only back of envelope estimations based on a billion assumptions. There's also so much dead code and now-unnecessary workarounds.
The front end has:
Front page
Index view
Catalog view
Overboard
Overboard catalog
News
Logs
And around 6 shoddily done mod pages
It's rather simple for a front page.
>>6626I've searched for a lot of headless imageboards and I haven't found anything decent.
>>6623
>>6633
>I'm leaning towards java though (likely spring / spring boot), being that its a popular enough language so that most people can understand it, but at the same time its typed. I know a lot of anons here really want to use a typed language
if static typing is all you want, it would take much less time to just fork lynxchan and rewrite it in typescript. rename your fork tynxchan or something. if you don't like the disorganized mess of mongodb, swap it for postgresql.
don't make the same mistake every overambitious programmer makes, ie., a phenomenon that has several terms to describe it: law of triviality, bike-shedding effect, reinventing the wheel, not invented here syndrome, etc.
>>6627
>If on top of that you use a language that is strongly typed, you probably won't have many security problems
famous last words
>>6663>you might as well just make new softwareif you're going to start from scratch, use rust w/ actix-web and sqlx
>However static typing does increase runspeed because you don't have to check types at runtime.rust is even more performant
https://www.techempower.com/benchmarks/>actix-core (rust): 651,144 req/s>vertx-postgres (java): 347,356 req/s>spring (java): 27,339 req/shttps://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/revcomp.html>Rust: 0.46 s, 499,024 b>Java: 1.54 s, 702,332 bhttps://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/regexredux.html>Rust: 0.78 s, 146,236 b>Java: 5.70 s, 656,328 b VIchan was forked from tinyboard, the earliest commits I can find from it are from 2011 or 2012. Laravel's first release was in 2011, Codeigniter was in 2006, Symfony came out in '05, Zend came out in '06, CakePHP in '05.
Literally any of these frameworks (even CI), and plenty not named, would have been better than the structureless, hand coded ball of spaghetti code that tinyboard/Vichan/lainchan ended up becoming, with its handcoded crappy ORM and barebones html templating, not to mention lack of a router.
Imposing some basic, even minimal MVC type architecture on tinyboard at the beginning would have made things a HELL of a lot better for all future maintainers even 10+ years later.
At this point coding a new imageboard from scratch is a huge task and few have attempted it successfully (lynx being one).
>>6834
My plan is to not give a fuck about APIs and just roll out your own for now. In most HTTP frameworks, that is trivial to change.
For now, this the general API I've come up with. Each endpoint basically assumes CRUD (create read update delete). This is based on the functionality of this software, which is the most complete of I know. My idea is to reach feature parity first, then expand functionality. This is merely a learning experience for me, so I don't think I'll end up finishing it. I'm passing on my notes.
Authed endpoints, all of them are prefixed with: auth/
Auth:
login/
logout/
Manage Users:
users/
users/{id}
Mod logs:
logs/
News:
news/
news/{id}
Board settings:
boards/
boards/{id}
Reports:
reports/
reports/{id}
reports/dismiss/all
reports/dismiss/IP/{ip}
IP posts:
IP/{IP} (eg. delete by IP accepts board as query param)
Bans:
bans/
Posts:
posts/ (no create, also this endpoint is suspect, more below)
Post File:
post/{id}/files/{id}
Threads:
/board/{board id}/threads (also suspect)
/threads/merge/{source}/{destination}
threads/{id} (also suspect)
This assumes that posts, boards, and threads all use global IDs. Specific board >> ids would still exist. I haven't figured out how yet, but I figure that working with global IDs will be more comfortable.
Suspicious endpoints:
I still haven't decided how to supply mod information when a mod sees a thread. I'm thinking of just using the regular non authed endpoints and checking for auth info there. If the user is authed, and has proper mod permissions, then get the info. Or perhaps just mirror the non-authed endpoints and make them authed. I prefer the latter one.
eg:
how to see a thread:
boards/{board id}/threads/{thread id}
and if you want the same information but also with extra info for mods, instead call
authed/board/{board id}/threads/{thread id}
Then for non authed endpoints, you pretty much have:
boards/
boards/{board id}/threads
threads/{thread id}
threads/{thread id}/posts
posts/{post id}
news/
logs/
I'm still undecided on whether to make it like that or to nest the resources like proper REST.
eg. to edit a post
boards/{board id}/threads/{thread id}/post/{post id}
instead of just
post/{post}
But anyway, this should give you a rough idea on the resources that would need to exist for a semblance of back-end feature parity with vichan.
We have like 20 tech users, so I realize the futility in reporting advances to my hobby project.
I took a break, but I'm back to working on it.
Error handling setup is mostly done. Now I just need to add errors whenever I make endpoints.
Auth was a pain in the ass. It's mostly done. Just need to write the logout routes. Everything is manual compared to ready-made auth endpoints from other frameworks I've used. That said, I am using an auth library that verifies JWT signatures and helps with authentication middleware, so it's not all bad.
Next on the list is adding more and more endpoints, which implies database changes, models, queries, error handling, etc. I decided I'd use a database modeler. That way the design is documented somehow. The program is called pgmodeler. It works really well. The pre-built binaries cost, but the code is open source. It wasn't too hard to build, luckily.
It has a "diff" function, so it checks my local database, and writes SQL to apply changes. This makes it easy to write migrations. Pic related.
I'm around 900 significant lines of code right now, which I calculate is around 1/10 of the way.
Just the handle_post() function of lainchan is 1k lines of code lmao.
>>6984
thanks for reporting in anon, other devs are interested in your work. are you the scala guy?
>pgmodeler
looks cool. I come from the oracle world so i had no idea this existed. Thanks
>I'm around 900 significant lines of code right now, which I calculate is around 1/10 of the way. Just the handle_post() function of lainchan is 1k lines of code lmao.
I have a feeling most of that is bloat, tbh its not surprising that with a more expressive and less boilerplatey language you'd have less lines. Plus all the poor architecture/spaghetti
One thing about your database, while splitting the boards, posts, threads, and files into separate tables is logical and common sense to the domain you realize to do metrics like showing the number of unique ips in a board, etc. you will have to be doing alot of joins + group by functions and possibly some window (over/partition by) as well, which will be pretty slow.
Maybe for reads on ip counts for example you could implement a materialized view or something, the problem is that ip counts typically display on the front page so if the query takes a ton of time with the grouping etc. it could make it vulnerable to DOS/refreshing.
Inserting a new thread - if there are constraints which say every post has to be associated with a thread, that means they will have to be inserted in both tables at the same time, like (pseudocode):
with insert1 as (<insert into threads table> RETURNING id AS threadid) INSERT INTO posts (id_thread <other fields>) VALUES ((select threadid from ins1),<other fields>)"
Pruning the catalog could be done with triggers on inserting to the threads table. Triggers could probably also auto lock threads with certain amount of responses.
your schema looks pretty good and logical, hope you don't mind if other people steal it :) not sure what the diff is between autosage and bumplock is thought?
just some suggestions again maybe youve already solved this
>>6986>are you the scala guy?Yep. Thanks for the feedback and encouragement, comrade. It is appreciated.
>you will have to be doing alot of joins + group by functions and possibly some window (over/partition by) as well, which will be pretty slow.That's true. SQL is really fast, but I guess once you start getting some serious numbers, thing could get slow. An easy way to remedy this is to de-normalize and put a board reference on posts. I don't have much experience with very high scale queries, not sure how non-performant this could get. In vichan, each board is it's own table, so you have to do a query for each table to count IPs, which I suspect is less performant than a single query with a group by.
>t could make it vulnerable to DOS/refreshing.I'm using redis and I plan on using it to cache certain results, metrics being the obvious one, but if possible, I'd love to cache shit like "all posts in a thread". I'm very much making this shit up as I go lol, I don't have experience with scala besides this, so I don't even know if this is possible with the libraries I'm using.
>Inserting a new thread has a mutual dependenceYes! Good catch. I bit the bullet and made post's references to threads nullable. There might be situations in the future where a reference to a thread might not be necessary (like associating posts with bans?) so I opted to relax that constraint.
>hope you don't mind if other people steal itThat would be very fulfilling. That's why I chose to use the PG Modeler, so that I could make nice pictures and others can use this work as reference/inspiration.
>autosage vs bumplock?autosage is when the thread reaches bumplimit. Here it's like 750 posts.
bumplocked is when a mod stops it from bumping, aka anchoring.
Update:
Users can now create threads and posts (text only)
This means most (important) non-authed endpoints exist.
This shit feels insurmountable at times……..
When posting, lainchan passes the request through at least these filters/functions, some optional and set in the configs, others obligatory:
- Validate captcha
- Validate bot obfuscation (field random data to confuse bots)
- Validate request has no referrer (block posts that contain HTTP_REFERER)
- Validate IP is not blacklisted by third party lists (aka DNSBL)
- Validate IP is not banned
- Validate IP is not robot muted (You get muted for x time if you post an identical message).
- Validate that thread exists
- Validate embed
- Rewrite fields if such a configuration is set (eg. force anonymous names so namefagging is not allowed)
- Perform upload by URL (ie you place a URL in the post and the server downloads the file for you)
- If no name is chosen, set name to one of the "anonymous" names. (in lainchan, you can set an array of names you wish to use instead of "anonymous").
- Validate that if the post is an OP it has an attachment.
- Validate post's body's minimum length.
- Validate that thread is not locked.
- Validate that the thread is not full (maximum number of posts in a thread, autolocks).
- Validate that the thread is not full in number of permitted file attachments.
- Validate the size of each file doesn't exceed max_size
- Validate the total size of each file doesn't exceed max_size
- Validate capcode if mod
- Validate joke capcode (setting to allow users to post as ## moot or ## hacker etc)
- Generate trip code
- Check for noko/nonoko in email field
- Validate filename extensions
- Generate filename using timestamp or a custom generator function
- Validate number of files doesn't exceed max_files
- Strip illegal characters from text fields
- Validate string lengths in each field (name, subject, etc)
- Apply wordfilters to body
- Apply country flags (using IP info)
- Apply user flags
- Apply markup
- Validate number of cites doesn't exceed max_cites (cites are >>{id})
- Track cites (validates each and stores them)
- Apply markup for cites
- Truncate filename to max_filename_length
- Calculate md5 checksum
- Calculate md5 of all checksums together.
- Check post for Flood (per IP) (eg limit posts by IP per minute)
- Check post for Flood (per file) (eg limit hash per minute)
- Check post for Flood (per body) (eg limit hashed body per minute)
- Check post using custom filters
- Apply extension functions (apply image transforms and create thumbnails (imagemagick+exif stuff), pdf, epub, txt, etc)
- If file has a thumbnail, OCR it and add it to the body (using hidden markup) to pass it through spam filters
- Validate image md5 is unique in board
- Validate image md5 is unique in thread
- If OCR was used, check flood and custom filters again lol
- Validate originality of body, mute if unoriginal (and config is set).
- Insert post into database
- Slugify post
- Insert post information to flood table
- If thread is cycled, delete all posts except last X posts and OP
- Update table of tracked cites
- bump thread (if not bumplocked, or full, or saged)
- handle noko
(async from here on)
- build thread's static HTML
- build board's static HTML
- rebuild any other themes that might be subscribed to the "post" or "post-thread" event (such as the homepage, overboards, and catalogs)
vichan/lainchan is incredibly configurable. Who gets to do what, filters, etc etc. I'm still thinking how to implement permissions. In lainchan, it's (mostly) done in the config files.
Eg:
$config['mod']['flood'] = MOD;
// Raw HTML posting
$config['mod']['rawhtml'] = ADMIN;
// View the report queue
$config['mod']['reports'] = JANITOR;
// Allow OP to remove arbitrary posts in his thread
$config['user_moderation'] = false;
I'd rather avoid that and have permissions be editable on some admin interface, plus shit like roles etc.
I was thinking of making everything be behind some permission framework, even if it slows down stuff. The default role would be "anonymous", so you could even make user-only boards. Say for example a secret /i/ board that is only available to those that have been given read permission to /i/.
With the coming of IPv6 and the increasing accessibility of VPNs, tor etc, it's becoming more and more difficult to ban spammers/wreckers. Ideally, sessions would also have "history". For example, if your session has completed a captcha in X minutes/hours, then you can post without captcha.
I don't know how to model this.
I'm thinking something like:
Board /leftypol/ settings:
Role: "EasyCaptchaSolver" Permission: "Post" + "Report"
Role: "MediumCaptchaSolver" Permission: … + "Post with image"
Role: "HardCaptchaSolver" Permission: … + "Create thread"
Role: "Janitor" Permission: … + "Soft delete posts/threads"
Role: "Mod" Permission: … + "Hard delete posts/threads" + "Ban"
Site settings:
Role: "HardCaptchaSolver" Permission: "Create User"
What is still TBD is how to implement this. I'm fine with storing this in the database and caching with redis, but I'm still not sure how to model the permissions. Apparently this is called a Role-based access control, pic related.
https://en.wikipedia.org/wiki/Role-based_access_control There's a difference between progressing and conforming. Most will keep the traditional elements but many are further diverged. I know one caused a stir of discontent in 2014 when they moved to a center-aligned view.
4kev is noteworthy too.
https://www.4kev.org/boards/design/>>9676.net vs today
>>10029
Java isn't my favorite language, but it's a better choice than the alternatives. The reason I didn't go with something like Php/python is because anything that requires low level functionality has to be done with some external C-built program, for example thumbnailing images with imagemagick, etc. Even rust doesn't have native thumbnailing. As opposed to natively with Java. Java also has 1.15x-2x slower speed of C so near native performance. Static type checking, mature ecosystem of libraries and tools that do anything you can think of, and it's widely known. Writing it in something like Haskell would get you hipster points but it also makes it a lot harder to develop. Additionally the thread per request process model of most java web programs, while less good than async style, is still vastly superior to the process-per-request model of php.
People have tried to write imageboards in languages like Clojure, Ocaml, etc. before and it ended up falling through because not enough amateurs out there know functional programming well enough to contribute.
The only other languages that could even be considered are C# (which I didn't want to do due to its association with the MS ecosystem and Stallman-brain people have here), JS/TS (which is associated with lynxchan), or maybe golang, although golang is also associated with a now apparently defunct other chan that was in development.
If I switch frameworks to be more microservice oriented I'll probably update that in this thread and let people know
>>10036>The reason I didn't go with something like Php/python is because anything that requires low level functionality has to be done with some external C-built program, for example thumbnailing images with imagemagick, etc.Why is that a problem?
>Java also has 1.15x-2x slower speed of C so near native performance.It's worse than 2x in most benchmarks and has to use several times more memory to achieve those speeds:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/java.html>maybe golang, although golang is also associated with a now apparently defunct other chan that was in development.How is that a good reason to avoid Go in favor of a worse language & framework? Also last update to gochan was just a few weeks ago:
https://github.com/gochan-org/gochan Hey /tech/ I'm here to say I'm probably going to be rewriting a chan in a new language/stack intended as a vichan replacement. I've reached out on lainchan:<br/><a href="
https://lainchan.org/%CE%BB/res/26674.html" rel="nofollow" target="_blank">
https://lainchan.org/%CE%BB/res/26674.html</a><br/>and I'm also reaching out here. Initially I was writing it in java, but after getting some feedback from other people I've decided to take a step back and solicit more feedback from the userbase and people here on /tech/ and in the wider alt chan community on what it should be.<br/><br/>What does /tech think of:<br/><br/>Architecture: Monolith vs Microservices<br/>Front end: SPA (ex: Angular, React) vs Server Side HTML templating<br/>Backend Language: Java, C#, Lisp, Rust, Golang, PHP, etc.<br/>Database: SQL vs NoSQL
>>10048
>except for Java
Why not java? it's got libraries for everything. And its faster than most languages except low level ones like rust/c/c++/golang. It's also got a low learning curve which is why many universities teach it as a first language which as a consequence now tons of people know it, and if they didn't learn that they learned C++ which is very similar to Java enough so they can easily make a transition. So the supply of people who know Java is pretty high and its easy to learn too.
Uber, netflix, airbnb, pinterest, instagram, spotify, amazon, slack, obviously oracle and even google search was originally written in java. Banks use Java, Universities, and fortune 500 companies.
Spring boot based services is a perfectly acceptable choice for any "startup". Its a jack of all trades language, I'm always puzzled when people choose the hype meme languages over a proven workhorse like Java, instead they say you have to do something crazy like write your site in an obscure dialect of lisp or erlang or something. You can't beat the rock solid stability, tooling and maturity of the JVM ecosystem and JVM itself.
Vichan derivatives have powered chans for more than 10-12 years now, imagine if the imageboard of the future gets written in a hipster language that is obscure in the future, imagine if vichan had been written in ruby on rails.
People's real complaint seems to be they want an "exciting" language to write shit in, but the older and more tired you appreciate an ecosystem/language that is mature and conservative and not constantly churning with activity and breaking compatibility creating more work every 6 months.
>>10059
>Use whatever language you are most comfortable with. Wrestling with the language instead of the problem at hand is a sure way to trouble.
Yeah thats the conventional wisdom but the problem is that to get buy-in from potential devs on an open source project you need them to want to develop in the language you've chosen. So actually the choice of language does matter in that regard because if its too hard of a learning curve they won't contribute because they can't/don't want to.
>>10069I guess for this I will do a strawpoll to get peoples approximate feelings on this.
I'm doing a language survey, based on the tiobe top programming languages, of what you guys think would be good to write in. Obviously, excluding non relevant languages like C/C++ assembly, and languages that will likely be used anyway like SQL.
https://www.strawpoll.me/45482737lets see what people think first off.
what would be the ideal social network site???
Imageboards (ex: 4chan, *chan “alt chans”):
Pros:
• Anonymous
• No registration
• OP requiring attachment forces content/memes to be produced
• Pruning catalog keeps server space needed to minimum
• Lack of upvote system reduces groupthink and disincentivizes non-originality
• Anonymity + siloing by board centered board identity over personality
• Linear reply structure that requires minimal interaction to read
• Effortposts that do happen are often phenomenal quality
Cons:
• Attracts schizos, racists and “free speech” people who have been banned from other SM for good reason
• Dated UX/UI (mid 2000s approx.) loved by users but retro looking and unappealing to “normies”.
• Dumb curation based on last bump time, incentivizing bait posting (probably this problem shared with other SM)
• Effortposts rare and disincentivized
Notes: Imageboards are in theory fantastic as the fact that they have a limit (usually around 350 threads per board) reduces hosting costs, and also incentivizes people to make interesting posts, because posts that do not garner replies will quickly sink and “fall off” aka get pruned from the catalog. This is also likely what gives chans their edgy character as a “provocative” or edgy post will be more likely to garner the replies needed to keep the thread near the top of the catalog. This can be a problem for quality as a well thought out long form “blog” style post may garner less responses and be far more effort than a 1-2 line “bait” post with an intentionally bad or controversial opinion. In this sense “trolling” and “baiting” are nearly built into the incentive structure of imageboards by definition. Some imageboards try to combat this by imposing a minimum character count for new threads (ex: 200 characters) but this seems like a bit of a hack and is contrary to the overall log
ic of the medium. Many of the “cons” of imageboards are pros in the eyes of long time users, such as its dated, non mobile friendly UI, and edginess, are seen as needed to preserve the status of imageboard users as outsiders. This creates a unique and culturally prolific space but at the same time dooms imageboards’ mainstream appeal.
Facebook:
Pros:
• Algorithmic timeline microtargets content to users, increasing engagement.
• Social connections keep people tied to platform.
• Makes a ton of money
Cons:
• Reveals personal information about name, job, location, gender, personal relationships status, friends, family etc.
• Incredibly personality centric
Facebook’s business model is pretty simple. Get people to sign up with their personal information and connect with friends (and maybe someone attractive you briefly met at a house party). Then take that detailed personal information and use it to microtarget advertisements at exactly the right demographic for advertisers and political campaigns. Mark Zuckerberg does not believe in privacy. Before facebook, people mainly used screennames and aliases online. It was facebook that changed this, so that people used their real names and faces online. Which is what he considers one of his big accomplishments. Needless to say, I hated this development and I’ve hated the culture it created ever since. Zuckerberg says he made the world more “open and connected”. Another way of saying that is more surveilled and more bothered by busybodies. Anonymity is freedom. There’s a reason every year young people leave small towns across this land to move to big cities, because living in a place where everyone
is in your business all the time and a bunch of gossiping people is a prison-lite environment. In a small town, even if you change, evolve, and become a totally different person, people will still remember and think about the things you did as a teenager or when you were twelve. I hate facebook because he’s turned the whole world into one big small town, from which there is no escape. People say the internet is making the world a smaller place, but has anyone ever stopped to ask if its getting too small? But yes, it does make a lot of money. And all they had to do was ruin the internet and possibly break politics to do it. Probably the only thing facebook does well is the algorithmic timeline which increases engagement.
Twitter:
Pros:
• Mobile friendly design
• Decent discoverability
• Hashtags
• Decent editor UI for posting
• Diverse OP content (images, animated gifs, videos)
• Pseudo-anonymity, except for blue checkmarks
Cons:
• Personality centric
• Meme ideologies
• Character limit incentivizes un nuanced “hot takes”
• Forced mobile style UI on desktop (“mobile first”)
• Nonlinear reply structure confusing, forced clickthrough
• Quote Replies
• Notifications mechanic incentivizes performative takes for subs/likes
Notes: Marginally better than twitter but still bad. Character limit makes sense from a mobile first perspective, after all who’s trying to read a 20,000 word thesis on a smartphone screen, however has the side effect of making people reduce their takes to the shortest, most un nuanced, un supported version of itself, which in effect reduces takes to “X bad, Y good” or sarcastic or cryptic one liners. Trump was a master of twitter due to his brain being small enough that his thoughts only require 240 characters anyway. Workaround is creating a twitter thread which is less readable than unbroken text and looks bad anyway, plus people have been trained by the platform to only read short takes and will tune out essay length twitter threads anyway due to twitter induced ADD. Non linear tree style reply structure is confusing and requires too much clicking to read through. The positive thing is the infinite scroll provides a constant stream of content, if the person limits their interaction to
scrolling and liking only (not replying or reading replies). Editor for posts is far superior to any non facebook platform. Quote retweet allow “dunking” and incentivize leaching off original content, “reply guys” who quote tweet someone else and add their own vapid or sarcastic commentary. Read somewhere that the twitter engineer that invented the quote retweet regrets it (as he should). Even regular retweeting is pretty lazy. If the only purpose of retweeting is outsourcing discoverability to the user, a better mechanism is needed. While imageboards incentivize creating content that gets a lot of replies, twitter incentivizes getting lots of likes, in fact a high reply to like ratio is considered a bad thing “getting ratio’d” as a reply with no like is a proxy for dislike. So while imageboards do not distinguish between positive and negative interaction, twitter and all other like based systems implicitly encourage “performative” content creation/takes a.k.a users saying the things that
they think other people will like, to therefore increase their own visibility, celebrity, etc. Rather than saying their honest opinion, if indeed they have any. Exporting Los Angeles style “trying to become famous” culture to the rest of the world. This is a cancerous dynamic. Should avoid like/dislike systems at all costs, esp. as complex feelings are reduced to a binary attribute (like/dislike). Possible side effect = polarization?.
Reddit:
Pros:
• Pseudo-anonymity
• Communities based on interest not personality
• Generally medium to medium-high quality content
Cons:
• Post History visible reduces anonymity
• Branching reply structure
• Terrible UI
• OP too short (as a “link aggregator” its often just a link to a news article)
• Likes/Dislikes, Karma, Gold, Silver, and the rest of that bullshit
Notes: Reddit is a website with a decent concept but a bad UI and worse userbase. Yes, as a channer the typical image of a “redditor” comes to mind. A limp dick neckbeard soyboy with a rick and morty t-shirt. Reddit is the Thomas Edison of websites, its users steal all their content from people that actually make original content, repost it, and then act like they invented the whole thing. A perfect encapsulation of middle class bland yuppie suburban consumer culture, a circlejerk which produces nothing original and never will. Part of the broader disneyfication and pg-13ification of a sterile and desexed pop culture, with a good dose of smarmy smarter than thou smugness and milquetoast center-left politics thrown on top. Peak midwit. Certain subreddits are niche and good. Reddit evolved to replace a lot of the old school bulletin board forums for niche gaming/auto/etc interests. At least its based on shared interests and not personality based drama. Like/Dislike and karma system encourages
the same hivemind and performative posting BS as with twitter. At worst as a “link aggregator” it simply acts as a comment system for articles on websites that don’t have their own comment section. Not much to be borrowed here mechanically, except maybe allowing users to make a http link as their op as an alternative to videos/images.
Blogs:
Pros:
• Pseudonymous option
• Potentially very high quality content
• High word count
Cons:
• Low discoverability
• Bad on mobile
Notes: Blogs and blog-like sites like Tumblr are actually surprisingly good. Unlike the transient nature of twitter posts, people tend to put effort into blogposts and even include sources from time to time. The only problem is that blogs tend to be separate websites and even when on the same domain (ex: wordpress.com) they tend to lack discoverability. Tumblr at its peak in the early 2010s seemed like a transitional website, a sort of nu-myspace. The downside of high character count is that blogs, regardless of “responsive” design, tend to read badly on phones and don’t lend themselves well to casual use.
Tiktok:
Pros:
• Extremely potent mandatory recommender system driven content
• Short video clips lend well to casual use
• Video format more accessible to dimwits and tired people than walls of text
Cons:
• Often cancer tier ultranormie content, on par with instagram
• Primarily consumed by children and teens
Notes: Tiktok’s content may be cancer but it’s format is genius. The most addicting app of the modern era. Instead of allowing people to search for content but adding recommender systems like youtube, tiktok leans in and puts it on steroids by making the recommender system the main way to access content. This is like increasing the nicotine content 3x.
Youtube
Pros:
• Audiovisual and audio content easier to digest
• AV/A content easier to listen to while moving (i.e. commuting or walking)
• Works for both long and short content
• Recommender Algorithm very good
Cons
• Video storage takes a ton of space on disk.
• Streaming content in a timely manner tricky, practically mandates CDN use
• Responses text-only
Notes: The second most popular social media network. Takes way too much storage for video. On the plus side the recommendation engine is scarily good and addictive. AV content can be consumed with walking, driving, etc. while reading text for that is pretty difficult. Object storage and delivery (files, videos, images) is a usually overlooked aspect of these sites. Responses being text only is listed as a con, but it can be a pro in the sense that it saves storage space and it encourages being OP-centric in an r9k-ish way.
Runner ups:
Linkedin – facebook for jobseekers, professionals, and hustle culture psychopaths.
Snapchat – fancy texts that delete themselves.
Pinterest – Not familiar enough to comment but looks like a glorified booru for middle aged women
Telegram/Signal/Whatsapp/etc. - Texting that’s ““secure””.
Medium – Blogging for people who are too lazy to set up their own blog. Often mistaken for legitimate “news”.
Why is there no facebook-but-not-evil?
Like I'd legit make an account on something like a facebook if it just fixed some major shittyness, e.g.:
- pushing random stuff at you that it thinks you will like in the moment, but no reliable access to notifications or a persistent list of things you might like, so you can check if of your own volition
- facial recognition, forcing you to use your phone number, and all that creepy super personally identifying stuff
- not respecting the users at all
- thats basically it actually
why cant we just have pseudonymous facebook that give you a bare minimum of respect in how it shows you things. Just a facebook that doesnt fingerprint you and gives you power over what you see, for your benefit….
I guess the big issue would be paying for it. Federated models are super unappealing to non-technical people when theyre expected to find or choose or set up server instances, and they can be not even good for a security perspective if everyone is using one main instance. But personally i think at least having the option to set up your own thing is a huge coup.
A centralized model would need huge funding, and who will give it? Companies who would just control the project and change it to suit them. No one donates. Maybe mine bitcoin in everyone's phone who has the app i dont know :p
P2P is ass generally and doesnt allow for the extra space to store stuff
anyways this is what I consider might be the best social media. A bunch of linked personal sites, with potential interests and stuff being recommended to you, that people consider normie and will use to organize stuff irl and advertize their business and do all kinds of super normie stuff on.
Just has to not track you, fingerprint you, demand identifying information, spy on you, has to be FOSS, has to allow weird connections like Tor, etc. Allowing but not forcing federation would be even more awesome. Also maybe encryption by default :p
>>10256
A theoretical true recommendation system would tell you, "if you liked that, then you may also like this" based on correlation clustering with other users data in the set.
The problems are:
1. Real recommendation systems don't work like this. That's the pretense, but they're either straight up ads driven by auction, hand curated by staff, or manipulated by various influential groups.
2. It presupposes that the centralized dataset exists in the first place. Are the recommendations worth keeping this centralized power around? I remember reading that netflix switched to either random or most popular, and getting better metrics than their best matching algorithm. Doubly not worth it.
3. It dulls people's organic search skills. They come to expect algorithmic recommendations, which also means dependency.
That's if the organic data hasn't atrophied away.
4. Even the best algorithm tends towards mediocrity. And how could they be otherwise. No algorithm can bottle human serendipity.
In the rare times I browse youtube in normalfag mode, I've watched popular/trending things. They are always take-it-or-leave it tier. They always have a junkfood aftertaste.
The decentralized version of algorithmic recommendations is NON-algorithmic recommendations. It's just a bunch of anons shitposting, and occasionally cool things surface organically. It doesn't scale, can't be replicated, and will likely only decline in appeal going forward.
>>10252
>Don't put "normies" in scare quotes you fucking normalfag. The arrival of social networks marked the beginning of the death of the internet by commercialization.
This is surely precisely why "normies" should be in scarequotes. A normie, a normalfag, whatever you want to say, is not a real thing. It's not an inherent trait of a person. Put geeks on Twitter and you get normalfaggotry, sit normal people down with only a HTML editor and you won't. Sites shape the people who use them far more than people shape the sites they use.
>>10260
tbh a true recommendation system seems just as, if not more philosophically objectionable than the various half-measures we've already got. non-algorithmic recommendations passed around by actual human beings and randomly searching by yourself are by far preferable.
also there is a fifth disadvantage which has developed concurrently, even if it's not an essential feature of a recommendation system: not only are people's organic search skills dulled, they are crippled. most sites don't support a ton of basic search operators (at best you tend to find that you can stick quotation marks around things to increase the chance they'll actually appear, and if you're very lucky, use asterisks as a wildcard. that's it.), often sites will have infinitely scrolling pages rather than pagination, ensuring a browser crash before you've found what you were looking for if it wasn't incredibly recent, especially in combination with the removal of basic features like being able to search for things from oldest to newest.
>>10287
We're losing the tech culture war though.
Freedom is "boomer".
Programming, hacking, and knowledge is boomer.
Wired Ethernet is boomer.
Torrenting is boomer.
Literally everything that can break dependency and control of the tech giants is boomer.
Selfies, wireless charging, proprietary connectors, being an influencer, streaming culture, no headphone jack, forced upgrades, and app stores are all the cool hotness, according to the capitalist controlled media.
>>10238
reposting comment from other bread, basically the best would be an all-in-one like Facebook as long as privacy and personal data issues are solved
Perhaps I am biased since it's been my main platform for years now but Facebook seems like the right idea for an ideal social media site.
Its biggest pro is really monopolizing and deciding they were gonna steal features from every other social media site. Completely absorbed Snapchat into its Messenger app, stole Craigslist's role as the local marketplace seller, fucking integrated its own Tinder dating feature, blogging exists through Pages and accommodates effortposts, it's much shitter now since that feature got whored out to corporations though. But that encouraged users to take advantage of Groups, which is basically subreddits. Won't find medium-high quality content as often, but if you curate your group to being niche enough, it can form good communities and attract the autist who'll effortpost and spark good discussion occasionally. Its Video section is okay, isn't as user friendly Tiktok which was built to host video, and its algorithm can't seem to curate content as good as YT- also hasn't attracted the quality content creators from either.
Also, I think it hits a good balance between anonymity and reddit's activity visibility. You don't have to give up any information on registration, you can make a completely false profile with a fake name, have almost complete control over who can comment or see the shit you share/post, and pretty much go into discussions with anonymity since people can't really search for comments you leave on profiles and pages. As long as you keep an iron lock on info FB network can cross reference like location or personal history, you'll never get found by IRL peers. Posting edgy bait on public shit is also discouraged since enough profile reports will get Zuck demanding to verify who you are, so that type of shit is discouraged(unless you find private edgy groups that don't report shit, which is actually very easy to do. Heard Proud Boys and other faschy types would just organize and shit through Facebook Groups, actually).
From what I know China's biggest social media WeChat also follows this all-in-one-model kinda, and I think Facebook is close to hitting the magic recipe in that regard, although it's personality centric on the profile side, it's really easy to just find pages/blogs or groups tailored to a specific topic or interest, so it lends itself to discovering/building communities easily. it's just fucking plagued by capitalist monetization and all the fucking breaches of privacy and demands for personal data, when they're not just stealing it covertly. Assuming a socialist revolution gets rid of these elements, basically an all-in-one model improving off what Facebook's built so far seems like the optimal way to go.
A discussion forum, assuming a socialist society:
1. Unique Internet ID that gets encrypted and is connected on the back-end to your real name. This way you can avoid bots and shills easier. Real name can only be accessed through a court order if criminal laws are broken. No TOR, no VPNs allowed. Only allow domestic IPs may post, foreign IPs can only lurk (no USano zone).
2. On the front end, you can remain anonymous or post with a fantasy-name profile.
3. Different boards/subforums, curated by the owners, with many general threads.
4. Sidebar with the most active threads, blog-pages that summarize or compile popular threads for the ultra-normies, ability to bookmark threads and posts. Threads do not disappear like on imageboards, although a user can delete his posts if he wishes so.
5. Obviously no advertisements. No upvotes or no likes. Threads that get bumped show at the top, and each board can sticky a few threads.
6. If a user opts to use a profile name, he can also opt in or out of receiving private messages.
7. Duplicate threads get deleted.
8. Some boards or threads are marked as serious - no memes, no emojis allowed, minimum character count. Some boards or threads are marked as fun - (almost) anything goes. Some boards or threads are marked as shitposting - anything legal goes.
9. Moderation depends on the thread and board type. Jannies get paid to moderate by the state. Serious threads have strict moderation, shitpost have very loose moderation that allows insults.
10. No blatant discrimination allowed, no racial or religious violence, no targeted harassment campaigns. Ableist slurs like retard are allowed and even encouraged.
>>10457
I've read retroshare didn't pass a security audit. And anyway, it is not anonymous, which is a downside for a supposedly anonymous imageboard, plus it tries to do everything - mail, chat, forums, VoIP, god knows what else. The same problem I have with i2p on java with its cluttered as fuck router that is also a mail server/client, torrent client, apache web server and whatever the fuck else.
I want a software that does only one thing, but does it well, ie provide imageboard infrastructure. Anonymity requirement limits to i2p overlay network, in i2pd router implementation as java router sucks. But the problem with i2p router is port forwarding for at least somewhat comfortable speeds, and this difficulty would discourage regular users from using the software. It needs to be a package that you install and it just works, or else nobody would bother.
>>10454
No, because think about it. An imageboard has increasing post numbers, and how would that be incorporated between multiple sites? Who's going to keep track of what post number the whole system is on? And if different sites track them individually, then how would posts be copied between sites? if the post numbers overlapped then which would take precedence? and if you change all the post numbers it would break all the links as well, correcting that on one site is a bitch, imagine correcting it on a dozen.
Maybe if instead of a No.# it used some sort of hashing/consistent hashing, but then it wouldn't look the same. How would cross-site linking work? Maybe it would have to prefix a domain, like >>>>org.leftypol.1234 or something? but then that would totally change the way links look, not to mention involve an ungodly level of cross domain image loading on the hover thumbnails, etc.
plus there's the fact that most imageboards have extremely edgy content and wouldn't want to necessarily trust another site enough to be in a ring with them, seeour situation with getchan and how they turned out to be pedos and we were only in a webring with them, imagine if getchan and .org had been federated or something.
The technical aspects of making a federated or general distributed system would fundamentally alter the nature of what an imageboard actually is and how it even looks.
In my experience people generally underestimate the difficulty involved in making truly distributed systems with no central authority, just thinking about it for a moment will bring up all kinds of challenges/design decisions which no one has even begun to think of.
All these "muh federation" guys should literally be forced to read a book on distributed systems before spouting off about how its superior.
I've attached one here. Tannenbaums books is also good if you can get it
Why vichan hasn’t been replaced and probably never will
Vichan and Vichan derivatives such as lainchan have long been the standard of imageboards in the “alt chan” universe. Briefly, Lynxchan looked poised to take over as the imageboard standard, but it’s reliance on mongodb left it vulnerable to data corruption and unreliability issues.
Any semi-competant coder can create an imageboard with 80%+ feature parity with vichan within a few weeks or even days. So why, after all this time, has no imageboard managed to take over from vichan?
Wordpress is blogging software written in PHP, from the dark ages when PHP was a terribly designed language. Like vichan, wordpress is a structureless, poorly written, outdated, spaghettified oldphp-based piece of shit. Yet, it persists. There are countless plugins, themes, and tutorials created for it, which enable countless nontechnical small business owners to create good looking websites with prebuilt themes. The very ease of use corresponds with it being hell to maintain and write plugins for. Ironically this self-induced difficulty has created a whole ecosystem of plugin and theme developers, selling overbloated plugins that perform functionality that could really be done in a few lines of code, mindlessly installed by nontechnical bloggers. Even conservatively, around 30%+ of all websites use wordpress, including major corporations like airlines, banks, and media conglomerates. The fact is, there is a self fufilling prophecy, where a software’s popularity can meme itself into existence. Wordpress is the Kim Kardashian of cms software: popular for being popular, despite an utter lack of any other redeeming qualities. And vichan is essentially the wordpress of imageboard software.
Like wordpress, there is a whole ecosystem of people who have become familiar with vichan+derivatives, their configuration options, their ticks, their idiosyncrasies, and have created an ecosystem of themes, scripts, and so forth. Not to mention a userbase which is used to the vichan style UI/UX experience and revolts at the slightest change. After all, that’s really what usability is: familiarity.
It’s entirely possible to create a piece of imageboard software that has feature parity with vichan, but it is much more difficult to create a piece of imageboard software which slavishly recreates all of vichan’s idiosyncrasies, ticks, warts, and all, which is really what people want: a drop in replacement. And since the whole point of creating new software would be to get away from some of the warts, the potential writer realizes that there is really no point in trying to begin with, other than maybe as some masochistic coding exercise. This is why lynxchan was really doomed from the beginning, whatever it’s technical merits or drawbacks.
The chans are dying. Really, they have been dying a slow death for quite some time, given a bit of a boost from the anti-sjw/Trump mid 2010s era, but quickly returning to a dying status, accelerated by the death of 8chan. Now, in the minds of normies, chans represent at worst, a hellhole filled with incels, white supremacists, pedophiles, and god knows what else. At best, they represent a “boomer” style of communication that’s deeply uncool, as opposed to the dopamine inducing, shiny-colored-box experience of tiktok and fbi.govs.
The conservatism and resistance to change of the current chan userbase is inherently connected to it’s age. While there are of course some young people, the preponderance of current alt-chan users are 25-40 year old millennials who came of age during the 2000s “golden era” of 4chan. This can be seen most clearly on lainchan, which is rife with arcane references to gibsonian cyberpunk novels, 1990s anime, ‘hackers’, and disgraced free software guru/activist Richard Stallman.
Like an aging video game with a slowly eroding multiplayer playerbase, the chan era is slowly passing from history. Many of the young, edgy people who would have been attracted to chans in previous eras are now on platforms like disc.ord. The whole appeal of php-based chans is that cash strapped youth could buy cheap shared hosting and slap a chan together with minimal technical knowledge and little money. Now, fbi.gov allows them to do the same thing with absolutely no technical knowledge, no money, and no effort. The barrier to entry is far lower, not to mention that a fbi.gov “server” does not expose the owner to the same level of legal risk as ownership of a standalone website. The private nature of fbi.gov servers also keeps it away from the prying eyes of journalists and others who would likely freak out if they saw the same or similar content on a website.
It’s only an influx of new people/users that would lead to the space for a radical reimagining of anonymous internet communication to succeed. As long as the chans remain a place for aging neckraging neckbeards, driven nostalgic for a bygone era of the internet, rather than a vision of the future, the chaniverse will continue to bleed users to other platforms.
To sum up, what this means is that there is really no chance for vichan to be displaced as the software of choice for chans since chans are dying. If chans were rapidly increasing their overall userbase you would see a proliferation of new and sometimes experimental software and formats. But you see the opposite.
>>11475>There just isn't anything more modern that can be done with this concept.there literally is you just dont like it, take a look at some of the screenshots in
>>11352
>It doesn't need le rounded corners.says you. Oldfags may not care but the autistic zoom zooms need their rounded corners otherwise they click off and the chan-userbase declines even more. Why the fuck are you so allergic to modern looking UIs?
>>11480I come here to catch up on the latest world news after a hard day's work lad
Go pester the zoom zooms at leftypol v 2.0 over at leftchan dot net for your rounded corners?
Imageboards are barely alive as a whole, let alone truly anonymous ones (i.e. tor/i2p)
Why's that? One practical problem is the lack of notifications. You have to go an manually check for replies, which is further exacerbated if you use tor browser (which you should) and have to remember every thread you replied to and check them individually.
The other problem, more on the ideological side, is the need for a browser. Browsers are bloat capitalismware, basically a proprietary platform of google at this point. And does it really feel nice to run such an immense amount of code just to display pages of formatted text and submit posts, in a highly ineffecient manner, both computing wise and usability wise?
I think we need an imageboard protocol. Different boards, one format for viewing AND submission, different clients, NOT depending on web (TM) technologies and providing notifications. That is one way. Another would be a truly distributed, p2p system, maybe like freenet forums or gnutella, but such a system is much harder to implement, and after all aren't freenet forums dead, and why?
Another option would be to (eventually) add federation between different servers, mirror posts from one another (would require some unique probably hash based post ids (which would admittedly be quite long)).
Thoughts?
>>11484>You have to go an manually check for replies, which is further exacerbated if you use tor browser (which you should) and have to remember every thread you replied to and check them individually. Notifications can't be implemented without user accounts, which goes against the low barrier to entry of imageboards. I wouldn't be opposed to making a chan that required registration but still showed users as anonymous publically.
As for distributed imageboard systems, some have been made but the technical issues would actually change the nature of what imageboards actually are as discussed in
>>11464Like the post said, anyone who talks of making a "distributed" imageboard needs to explain how it would deal with these basic issues of how site mechanics would work or translate from traditional chans to a distributed paradigm
>>11485yeah but that doesn't explain why reddit is popular compared to chans, since it's mostly the same in that regard. The only exception is the occasional AMA
>>11489As stated, if you're going to start storing tons of data in session/cookies to make up for lack of accounts, you're halfway to de facto implementing accounts without calling it that, with significant technical overhead needed to track all this stuff, all so people can avoid having to press the login button. A better solution is to implement accounts but allow people to comment anonymously as always, however, put additional features on accounts like notifications, saving preferred theme between sessions, maybe direct messaging, hell you could even get more accurate metrics on the true amount of users. It could also help during raids if you disable anonymous posting temporarily. Also you could require TOR users to use an account that way you don't just have to resort to banning the entire TOR node, and can allow legitimate users to continue using it. OFC the spammer could always create another account, but nothing's perfect.
CAPTCHAs and other anti spam measures could be pushed off to the account creation stage, leading to better UX for the user instead of making them solve a captcha any time they want to make a thread or w/e.
In fact a ton of the spam problems of chans/imageboards could be solved with accounts, merely by having them as an option.
>>11484new protocols for everything is a complete and utter waste of time.
to get more people using imageboards the number one thing to do is identify areas where a discussion space is needed and then create one using imageboard software, without carrying the stupid baggage of chan culture with you, and then trying to create a board that best meets the need of the resulting community.
imageboards are no more difficult to use than newspaper comment sections in principle, the problem is purely one of application. fundamentally there's no reason you couldn't use lainchan to host a website for old women to discuss knitting patterns. sure, they'd probably actually enter their name and email (oh no!) unless you got rid of those fields, but you could certainly do it if for whatever reason you had contact with enough old women to get it started.
a new protocol wouldn't help. an imageboard with its own protocol is either something to ignore or (if by some miracle it actually had enough users to be worth caring about) something to access via a browser addon that the scum at mozilla will keep intermittently breaking on purpose because they hate all that is good and holy and know that they can get by with being marginally less evil. browsers are their own hellscape, but "make a new protocol" is a false dawn for breaking out of that. at a push, it works for large tech companies who're probably going to have the application to access their protocol just be electron reskinned-chromium bloatware anyway.
>>11489>>11490What a sad state of things we have in 2021 that people don't even think about implementing stuff outside the browser…
>>11491> something to access via a browser addon Again, why?
>browsers are their own hellscape, but "make a new protocol" is a false dawn for breaking out of that.Ok what other options are there? By your own protocol I don't mean stuff like freenet or bittorrent, but at least a standardized json api for different server software (presentation layer).
>>11487>>11491The gemini people made a new web-protocol and it's working for them
maybe the trick is to make it very simple so that it's easy to maintain ?
>>11497yeah, if you dont allow namefagging all the namefags will just sign their name with
-le namefag
boomer email style.
>>11504No it isn't. You lose some benefits and issues of the anonymous imageboard, but it's not like reddit at all. A forum, maybe, but not even.
Look at 4kev for an example.
>>11511A perfect backend would be storage agnostic.
abstract class StorageProvide
FileStorage
S3Storage
Not sure what others make sense. Amazon is not cheap, and requires some hoops to get an account. So requiring people to use Amazon's services might be a non starter.
>>11512most cloud hosters provide an S3 compatible object store, even low tier ones like digital ocean and OVH.
Even if they don't most open source object store software like minio, ceph, riak, etc. uses or at least provides an s3 compatible API. The only reason corporations don't use minio is because the free version is AGPL licensed, but thats not a problem for us
>>11527My bad, I was looking for something like
https://docs.microsoft.com/en-us/sql/relational-databases/blob/filetables-sql-serverRegardless, the server could store files on any filesystem, including ones that directly map onto an existing database format.
>>11532If you wanted to eschew transaction safety, you could store the data on a network share (NFS, SMB, or a 9P files server), or a regular filesystem while a daemon synchronizes multiple servers (using rsync, or a VCS over ssh).
Using the filesystem as the storage interface of the server allows it to access anything you can mount.
>>11533rsync could be ok if you have 2 servers, id hate to scale that to 3-4 app servers. There's some tool that uses rsync to do live real time updates as a daemon, I forget the name though.
I've always had problems mounting storage from multiple servers.
The essence of a chan
This is a followup to the “ideal social media” post. I’ve been thinking about site mechanics and how they influence behavior. I’m trying to drill down on what a chan is, really at it’s core. In silicon valley, they often do these “[site concept] in one SQL statement” type overviews.
To analyze a site/app we have to think about the behavior it “mechanically incentivizes” i.e. how the actual mechanics of the site/app force or incentivize the users to do certain things and behave in certain ways. The essence of reddit is the link. Reddit is a link aggregator which means the whole site revolves around the posting of links. In this sense the very behavior that reddit is so hated for, stealing OC, is literally the basis of the site. The basis of twitter is saying shit people like. On twitter, people are rewarded with likes. Having a post with more replies than likes is called “getting ratio’d” and is considered bad. However on a chan the incentive is the literal opposite, more replies mean the post is bumped to the top. So “taking the bait” is literally built into the mechanics of a chan. While the twitter way encourages a “Los Angeles Hollywood” style mentality, of performative takes to get likes and followers, doing and saying things you think other people will like, chans mechanically incentivize saying things people will respond to. Also chans have a group identity based on boards or the site itself.
chans/imageboards are inherently filesharing sites. They revolve around the sharing of files. But the files themselves are small, and often original content. Unlike, say the pirate bay which concentrates on sharing large files of decidedly not original content (movies, etc).
Every OP on an imageboard is essentially a file or collection of files, with an optional comment, and some metadata for discoverability. What separates it from something like instagram is the anonymous nature of imageboards, transient nature, and lack of personal identity.
So a chan must:
• Be Anonymous
• Emphasize activity/responses over popularity
• Emphasize originality over derivative content
• Emphasize group identity over individual clout
• Be filesharing-centric
• Be Transient
In order to discover the true essence of a chan we must ask ourselves, what if every aspect of this was hyper exaggerated to the max.
Anonymous – Anonymity serves several purposes. Contrary to popular belief, “free speech” is not one of them. While free speech is good, it’s not inherent to a chan, nor are non-chans inherently unfree. What speech is allowed is a consequence of the moderation of the site, and you can check various alt-tech platforms or dis.cords to find content that’s just as edgy as that of a chan, if not more. What anonymity does do, is prevent individuals from accumulating clout (or money) on the basis of their statements. Since everyone is anonymous, and there is no “follow” or “subscribe” function, this means users are not mechanically incentivized to say things they don’t mean, just for the likes/followers. Everyone is equal, everyone is just an anon. The main value of anonymity isn’t free speech, it’s anti-celebrity. In order to be even more anonymous, all namefagging and tripfagging should not be allowed, in fact the name field, email field, and tripcodes should be totally eliminated. Additionally (you)s should also not be allowed. Notifications are fine but should simply state that there are new replies in a thread you posted in, not if they specifically replied to your post.
Emphasize Activity – One of the most neglected aspects of site/app mechanics, but most important is bumping, or the order in which posts and threads are displayed. Twitter displays tweets by creation date of the OP. This works well in incentivizing people to make new threads because old tweets get left behind and people lose interest in them. However, the incentives on chans are to make threads that get a lot of replies. Twitter incentivizes more content, but Chans incentivize more triggering content. Due to the catalog limit, this leads to a sort of collaborative sorting, a brutal darwinian ecosystem in which the boring posts sink to the bottom. I’m not sure which is better, twitter create date sorting or sorting by replies. Sorting by last reply incentivizes edgy posting, but also baiting. Also cyclical threads just end up become glorified chatrooms, sometimes with namefags to boot. Are “generals” really a good thing? Or should people just create a new thread for every single topic. If people are still stuck on the “a thread died for this” they can just increase the catalog size from 350 to 500 or something. Or change the formula to a time-based formula, like have posts last 2 weeks.
Reddit uses a more complicated formula for whats displayed, including posting time decay formulas, likes, etc. I don’t think a more complicated formula would be good.
Maybe its better to stick with the last bumped formula, but twitter style has its advantages too. However ordering by create date of OP would mean that shit like saging, cycling, bumplock, etc. would essentially be obsolete since posts would not be bumped at all.
Originality – R9K enforced on all non archived posts, both imagehashes and text/comment. Literally to the point of putting a unique key constraint enforcement at the database level. In addition to showing number of replies, images, etc. threads should also count/show the number of effortposts, meaning 1400 or 1500+ character posts in a thread. Autoarchiving of all threads, as a setting per board. So for example autoarchiving could be turned off on /b/.
Front page would be overboard.
>>11538>all namefagging and tripfagging should not be allowedPerhaps people would start to sign their posts or pgpfag, but it allows for more simple post storage.
>Are “generals” really a good thing?Generals have plenty of activity, so maybe they should be substituted for their own if slow board. With hierarchical boards, country specific boards would not clutter the top directory.
>putting a unique key constraint enforcement at the database levelThe server could store data on something like a venti filesystem
http://doc.cat-v.org/plan_9/4th_edition/papers/venti/Posts could also just reference files by a hash and upload them afterwards, if they don't already exist, allowing to seperate both actions without identification. Tags should be enforced on file upload, so the server may act as a booru and prevent image duplication.
>>11540>Posts could also just reference files by a hash and upload them afterwards, if they don't already exist, allowing to seperate both actions without identification. Tags should be enforced on file upload, so the server may act as a booru and prevent image duplication.AFAIK wordpress operates similarly in that you can re-use uploaded images between posts. It sounds good but the only problem I can see is that people might be too lazy to upload files separately.
Basically you would be combining the functionality of a booru and a regular chan
>>11541The seperation of posts and images would be at the protocol layer.
It could be made invisible at upload, essentially looking the same as vichan, except for the tagging and faster upload, when a file is already present.
>>11546>may lead to less discoverability for shitposts and niche topicsI mean they will still show up on the overboard, right? Plus you could have multiple tags so have them in one mainstream tag and whatever niche tags they want.
Hierarchical board based is an interesting idea, the SQL for showing it could get pretty complex though.
you should explain hierarchical boards more though
>>11547>you should explain hierarchical boards more thoughTo detail my approach to a hierarchical imageboard:
I'm working on a non-standard http server that translates POST requests to writes instead of CGI execution. By default the server would have read, write and exec permission on directories, read permission on all files and directories would be created sticky.
Mounted at the root of the server would be a filesystem, that passes through all writes and on reads generates static pages on the fly (or with some form of caching, idk yet).
Boards would be directories not starting with a special character stripped by the client, otherwise they would be threads.
Either posts would reside as plaintext files and have a header with file references. In this case the client would write to that file.
Otherwise a post would be a directory without write permission, with a plaintext body file and fields or references as separate files, which means it could not be directly written by the server, instead being written as an asynchronous request to a fifo, which has higher throughput and is more secure than a CGI request.
>the SQL for showing it could get pretty complexI hope this makes clear that no tables are involved in the hierarchy. Administrative tasks would be handled by communicating with daemons through fifos or direct file/directory modification.
For bumping, threads would always be named by their last reply and the catalog would be like 'ls' in descending order. Anchoring and pinning could be done by adding special symbols to the thread name.
>>11548>could the hierarchical concept be combined with tags and create a hierarchy of tags?I guess the motivation for this is, that tags are more flexible than hierarchies, yet this may be corrected with symlinks, potentially created by a request over a fifo.
>I'd hate for the app/user to have to parse a tag/board tree N layers deepOnly a client directly accessing the file hierarchy would suffer from this. A special-purpose IB 9P server could be written, where mounting an overboard compiled a list of threads with limits set by the client.
>could users create these tags or would it be janny based?User creation of directories would be ideal, but small servers may not want this and would set file permissions accordingly.
>>11549>decoupling the pruning logic from the tags/categoriesIIRC the main motivation for pruning the catalog was storage constraints.
Because storage availability has changed, you have the right approach, that is probably to prevent necrobumping, and it could be implemented by a daemon deleting based on directory modification time, perhaps even inactive board hierarchies.
>>11550Interesting ideas although I was looking for something more in line with a traditional web app. Good luck with your project though, keep us all updated.
Also what do you think of the trade offs of ordering by last reply vs Twitter style ordering by create date with most recent first
>>11645ok, so go fucking use tick tock and leave here
if braindamaged kids can't read shit there are many platforms for them to continue damaging their brains on
>>11933>post history, ban appealsim done with these now
>word filters, and a site settings menu.decided against these. There aren't enough settings to have a settings menu and im against word filters on principle due to them being abused to force memes, and overused in general. If people really want wordfilters they can easily implement it in 1-2 lines in the comment processing
The future is syndie (
https://syndie.de/). Stop trying to make another chan which is brought and sold at your expense.
>>11970>Stop trying to make another chan which is brought and sold at your expense.uhhhh. what?
and can you explain that thing? TLDR don't just link an obscure website
Some criteria that could be used for new software:
—FOSS license (AGPLv3 is best)
—Minimal dependencies, use standard and widely trusted stuff like FFmpeg, ImageMagick although even they should be optional
—Generic SQL socket connections
—Sane default configurations
—Easily deployed on a new box by a script
Wakaba [
https://wakaba.c3.cx/releases/Wakaba/] basically fulfills these, except for maybe the license (it has been used as a codebase for closed-source software like 420chan's Taimaba), and it can take a lot of tweaking to work properly at first, as well as being a hassle to transfer to a new server box properly.
I'm having a terrible time at work, so to cope I've been slowly working on a new design of an imageboard again.
I scrapped basically all of my previous design. It was very silly to start with the API.
I don't know if it will ever get to a stage where it will be presentable enough. Maybe if I don't find a new job I'll end up finishing the design lol. If I had to put a number, I'd say I'm around 10% of the way done. This time around, I'll also try to resist the urge to program before finishing the technical design.
Again this is mostly for fun and to feel productive since work has been terrible. I'm also not with stable internet and sometimes no access to my non work computer, so I might drop it soon again. Lets see.
>>11645Agreed. Its not only zoomers. Its everyone.
>Only prisoners have time to read, and if you want to engage in a twenty-year long research project funded by the state, you will have to kill someone. >>13643I know one that may tick all these boxes but the community and staff try to distance themselves from 4chan culture - it was originally made by /jp/sies, so I don't know if it'd be bad manners to post it.
Also their git seems to be down at the moment because they recently updated everything to Python 3.
>>13649part of the problem with chans is that chans are intended to be small sites hosted on cheap VPS's so that mean that all of modern cloud based "system design" ex: using AWS features, even something as simple as an S3 Bucket is not allowable because muh AWS (and possibly for good reason of deplatforming edgy chans).
Accountless also nullifies a bunch of the security and authentication practices that modern frameworks take for granted like JWT or w/e
>>13663I really dont see why cookies are bad since they allow people to view notifications if they accidentally close the browser anyway
>>13666>I really dont see why cookies are bad Are you self-taught?
With localstorage the client does not need to communicate with the server so you don't waste of bandwidth and they have a 5mb limit compared to the few kilobytes cookies have. If you're going to store what threads the user made it's only logical that you have a jsonified string in the browser's localstorage. There's also the obvious issue of privacy here…
Switching from cookies to localstorage for fucking
notifications would literally only need a few lines of Javascript code.
>>13668app is serverside rendered, JS is only being used to prettify things. So actually using localstorage would require a rewrite in a front end framework or library.
>Switching from cookies to localstorage for fucking notifications would literally only need a few lines of Javascript code.ok then the source is linked here:
>>11958 please point out where the few lines of code are that would need to be changed
>>13664>>13687I can't vouch for or against them but there have been attempts at federation protocols for imageboards.
vichan has an experimental NNTPchan compatibiliity.
There's also a (Fediverse?) imageboard net, see >>>/meta/14325
>>13693A dedicated imageboard protocol would transmit less information, though a browser with good caching largely negates this.
The main motivation would be getting off the bloated web, to add more ui components without javascript and make a scriptable client. We could literally have things like a moderation bot moving WSWS posts into a containment thread and removing duplicates.
>>13888>>13857I think most people that take issues with Java take issues with the language itself. There are many languages that run on the JVM and the complaints are not usually about the performance of the JVM. As others said, if performance is an issue the GraalVM is an alternative, with ~10% reduction of cpu usage depending on your software.
The switch to openjdk is not relevant anymore. As the otther anon pointed out, basically any new software using the JVM is using openjdk now.
Once you filter languages based on, not being PHP, having sizeable communities (meaning maintained production-ready libraries and frameworks), cross platform, etc you're not left with many languages.
I for one dislike dynamically typed languages. They can be fun, but for production purposes, I don't see the point. And memory management must be automatic, so that leaves out C.
So you're left with C++, C#, Java, Typescript, Go, Rust, Kotlin, Scala.
Kotlin is improved Java. I don't see a point of using Java over kotlin, so fuck that shit. C++ is archaic meta programming magic smart pointer shit, hard to parse, easy to get wrong, so fuck that shit. C# is now cross platform but there's no point in using it over kotlin. C# is basically nice java anyways, and java has a myriad of more libraries and frameworks. So fuck that too.
Anyways, after that screening your left with:
Typescript, Go, Rust, Kotlin, Scala.
Go is sanitized C++ or safe C, whatever the case, the pros is that its a very small language. I don't see the point in using it over a more feature-full language, but that's just me.
Kotlin is a worse Scala. The only reason Kotlin makes more sense is if you don't know Scala. Even using scala as a Java instead of using more functional aspects make for a better experience.
Typescript is a major cope built on top of Javascript.
So I would use Scala or Rust, and since I don't really know rust, I'd just use scala.
As a minor plus, scala can compile to Javascript.
Scala still sucks, but hey, at least it's not Java.
>>13889Scala uses outdated versions of the JDK I've had trouble using lift framework on anything above jdk8, also Scala is too big brain to understand your average channer-scripter can barely understand OOP as evidenced by the shitty state of the vichan codebase let alone shit like currying and algebraic data types
honestly id rather just use Java since most of the FP features of Scala have already been incorporated back into bog standard Java. Java streams API already lets you do basically any type of FP you want as long as its with a collection. Scala is just a slower compiling Java at this point, I feel like the FP language hypers are still living in 2013 or something the point is you really don't need FP languages any more since Java/C# have already incorporated most of the major innovations of it.
If we write it in Scala/Kotlin then thats ok by me but personally I don't really see any reason to go beyond Java.
There's also the readability argument. The fact is that people esp. hobbyist coders love to bring up terseness as an advantage (ohh, look at le quicksort in 3 lines of code in haskell or whatever) and verboseness as a disadvantage (muh boilerplate). The truth is that verboseness is a feature not a bug, the question of whether a language is good is not whether its easier to write but whether its easier to
read, and can be read and easily understood by a large team of people with varying levels of skill, ability, and experience. That's the difference between coding as an engineer/architect and coding as a hacker. The point is the vast majority of time spent on a program will be READING the code not writing it and verboseness to a certain extent makes it more self documenting. If terseness was the end all be all then obsurantist Perl one liners with long ass regexps written by a sysadmin 10 years ago and never understood by anyone since would be the height of programming.
The fact is theres a lot more than just the language its the language ecosystem and related tools, tools like IDEs, automated testing and QA, deployment pipelines, etc.
I think people really have to ask themselves are they trying to make something that is there to be a workhorse or are they choosing a language based on whats going to be fun for the developer to write. I'm not saying you can't do this in Scala but there is a ton of documentation and best practices already established around doing it in Java. Any obscure library, any tool like Openio or some EIP pubsub system apache storm or whatever, rabbitmq, literally anything out there has an example and a library in java. If you choose another functional language, even jvm based youll often find something that was a wrapper around an existing java lib and was last updated in 2015 or something forcing you to go dig thru the librarys code yourself when something goes wrong only to discover its a bug that will never be fixed because the developer is MIA. And its usually nothing to do with a problem that can be fixed with a cute mathematical FP trick, its usually some bullshit around file/network IO, image/video processing, or other obscure shit.
Again I've written chans in a functional JVM based language before and encountered all these problems and more. Finally theres a case to be made that most devs, esp. self taught ones, but even ones who went to uni understand OOP better than FP because most people dont think in mathematical functions, OOP is closer to how we think of things in real life and thus easier to model, and also most university CS/ICS etc courses teach and give people years of experience in OOP and will cover FP for one class on PLT if at all.
>t. had to learn Scala, LISP, FP etc. in a graduate class on programming language theory and written a chan in a functional language on the jvm >>13899>scala>currying and algebraic data typesI wish.
>Scala is just a slower compiling JavaNo… I wouldn't even say that about Typescript vs Javascript, or Kotlin vs Java.
>he truth is that verboseness is a feature not a bugthe absolute state of
// in org.javachan.src.main.public.core.domain.external.joke.java
static final public <ISetup extends Comparable<ISetup>> ISetup javaBeanJokeFactoryProducer<IJokeElement>(ISetup setupJokeConfiguration) {…}
// in org.javachan.src.main.public.server.static.injector.store.adapter.Dispatch.java
@Inject SetupJokeConfiguration $setupJokeConfiguration
JavaStatementProducer jokeProducer = javaJokeProvider.javaBeanJokeFactoryProducer(setupJokeConfiguration);
jokeProducer.produceJoke(tracer); // outputs "java developers"
TBH any language you choose is fine, as long as it's not PHP for jesus's sake.
>>14249>pic 3lainchan's db
>pic 1Something I was messing around with. Just to get an idea of the problem.
>pic 2Gochan's ERD
I can't be bothered to figure out what the fuck lynxhchan is doing. Nosql is cancer.
>>14256arisuchan.jp is down and
https://arisuchan.com/ is non-operational. Some anon at wirechan says it's being set up
https://wirechan.org/g/res/161.html#q898 >>14256>>14257arisuchan was alive? I just kind of assumed it was dead for half a decade
>>14258Kind of this.
Imageboards (like forums) are a niche, but I don't see them going away any time soon, and not even for cultural reasons. They are (often):
- easy and low cost to setup and host
- low/non exploitative (good for people who hate 'big tech'/spyware/capitalism/'corpratism')
- convenient to use (no registering)
- anonymous (different conversation dynamic to identity-based social platforms)
Those alone will keep thousands of people or more either using it or some similarly-featured replacement.
>>14259what a dumb post
>>14251anon
tinous if you want to write in Scala you should use play framework or something else oriented towards monolith development not be fucking around with raw akka (play uses it under hood afaik) or http4s which is intended for microservices, you're unnecessarily increasing the workload for yourself, since we all agree people deploying chans typically aren't that technical and aren't going to be doing a microservices deployment, its hard enough to get them to deploy a restful backend+frontend let alone N different microservices.
>I can't be bothered to figure out what the fuck lynxhchan is doing. Nosql is cancer.tbh mongo wouldn't be that bad the only problem with document store is that the data can easily be corrupted due to no schema enforcement and migrations are a pain in the neck.
>>142672003? Most imageboards look late 00s early 10s. It's not the aesthetics that keep people off I think it's that net culture has changed too much. imageboards were a fresh alternative to forums. No registration, no reputation, no bullshit. However Reddit outcompeted the forums popularizing meta-sites where you can make a "community" with no hosting required. I don't think imageboards had a good answer to that, and 8chan was the last hurrah before the slow slide into obscurity.
>>14271All you're going to to do is create another incompatible API. Vichan json API is mostly compatible with Yotsuba. Lynxchan and jschan decided they wanted to do it differently for no reason.
>>14351I'm not a real oldfag and even I know that the Yotsuba most modern boards are based on is the redesign.
>>14359Fair.
Never use Java.
>>14350People have been saying this since Mootxico. It never pans out. 4chan might not be as popular as it was in 2016-2018, but it's way more popular than the years before - and it was already unusably crowded at that point.
>>13899>The point is the vast majority of time spent on a program will be READING the code not writing it and verboseness to a certain extent makes it more self documenting.It is genuinely worrying that you think this. I have no academic background so can't speak to the technicalities of this or that language ecosystem beyond the ones I have hands on experience. However, just from over a decade of reading other public codebases, I can assure you that verbosity does absolutely not ensure self-documentation. If anything, it often leads to obscurity by verbosity.
Readability, etc. depends on the programmers, or the maintainers - and let's be real: imageboards are filled with middling-at-best programmers, who blaze ahead with no plan or change them constantly, or simply have edit wars over petty disputes. This should be kept in mind for any public forum based project.
>>17993Upon further consideration, I think for just software design cloud systems architecture, ci/cd, deployment, itsm, and SRE could be cut out because all of that shit is really more the responsibility of the operators, not the devs.
the only traditional operator task i think maybe should be observability, that should be built in since its hard to do without observability having been an architectural goal from the very beginning.
>>17986It really doesn't matter which language you choose as long as its not a scary language. Eg. Scala, C, C++, or any functional language really. I don't see the point of java if kotlin exists.
>>17989Regarding the last block in the design, most chans will be deployed in a small VM instance, and that should be taken into account. Also monitoring and instrumenting is really valuable, so that's good. Something like opentelemtry.
The database and api design, plus the implementation are both the "same step". They inform each other at the design phase and will likely be implemented together.
>>17995yeah, I got busy and couldn't finish a response. Here it is half baked.
are we really wedded to scala? AFAIK Scala doesnt have a good API test framework like this:
https://rest-assured.io/unless someone here writes one. OTOH you can use rest assured with Kotlin. Kotlin seems to have alot better interop with existing java libraries.
There's a pretty good choice for all the people here saying they want jvm: CLOJURE
1. its a lisp, which floss and /g/ people love (scheme/cl)
2. Its more "purely" functional than Scala
3. It has way simpler syntax than Java or any other non lisp language since knowing () is literally 80% of the battle.
4. It has java library interop which is way easier than any other jvm lang other than maybe kotlin
5. It's a dead simple and a hellofa lot less big brained than scala, makes FP super easy for mere morals
6. Development cycle is faster due to it being dynamic, but still gets the benefit of AOT byetcode performance, once you're done developing/using the repl you can compile it to bytecode for production
7. No complex type hierarchies
8. Tooling is pretty good
9. No OOP option or object-functional, FP is literally forced.
10. Typical clojure style roughly corresponds to ML/OCaml with LISP syntax/expressions + hygienic macros, so you have to really TRYing to not write idiomatic code.
11. Good concurrency model (STM)
If you're writing in a domain where correctness is necessary (finance, embedded software) then static typing is a must. But for an imageboard? pfffff
Clojure is a no brainer, the only problem is that it's obscure, but trust me, when the entirety of syntax you have to learn is () most people can catch on pretty dang quick. Well ofc there are also vectors which use square brackets, and {} for edn, etc. but thats not that big a deal so maybe () is 80-90% of syntax rather than 100% like with other lisps. Still extremely simple. and the best part is you don't have to be some guy with a PHD in category theory.
I checked today and leftypol seems to be cached rapidly, what is the decision making process that decides if a website is to be actively monitored and cached?
>>18067It's a very simple language to the int anyone can pick it up very easily with marginal experience programming.
>>18143They are, like I said
>>18142The only people who begin to care about imageboards anymore are morbidly curious tourists and obsessed reactionaries, the first of which couldn't care enough to design newer software, and the second of which don't tend to have any real skills or talents or any drive for anything
Current imageboard maintainers like leftypol's devs will only go so far as to do small maintenance and feature additions to an existing software base
Also non-tech-wise:
Imageboards just suck
4chan always sucked even early on
It helped drain the mystery out of the internet
4chan and its main English language predecessor, Something Awful, more than anything else helped create and propagate the mythology of the implied straight white male internet dweller, with a repetitive "meme" vocabulary as a system of signals for it
Even to this day, stuff like QAnon and soyjaks spread throughout the Internet like a miasma wherever people tolerate it, taking up as much space as possible, killing off any discourse remotely interesting or novel
So, yeah
It's a dying space that was never really any good in the first place
>>18147Yeah
As it stands, every post is just a single self-contained <div> with its own media attachments and whatnot, which lets drive-by posts happen pretty easily, as well as dump threads, passive-aggressive responses that don't directly reply to whatever they're addressing, etc
Threads need to be "smarter"
They need to have coherent semantic links in some way
>>18154what's not good enough about threads and >> links that they dont qualify as coherent semantic links?
To me the fact that every post is its own thing just makes it easier to visually identify individual posts. It's not very different from any other social media or chat (except for threaded/tree structured ones, which tbh just make it hard to see connected comments, only thing that makes it usable is collapsibility). I doubt the underlying html actually encourages drive-by posting and dumps, just appending the posts to the html in one div would allow the same exact thing with an uglier interface.
I have to learn Go. Might make a pet imageboard project in Go and drop it before I even finish 10% of it.
>>18005The category theory stuff is overrated. You really don't need to know much. Just practice. At least with scala.
In fact, after doing Scala for 2 years, I'll conclude that category abstraction libraries like Cats and CatsIO are actively harmful in actually using said abstractions. Either the language was not made to accommodate these category theory abstractions, or the Cats/CatsIO library sucks balls and was implemented wrong. I have more gripes with Scala 2, but they are too abstract to explain, unless you've had to wrestle with the compiler and type checker. Shit like "tagless final" for libraries sounds really cool, but whomever is still pushing that shit today is the devil. Namely, doobie fucking sucks, but it's one of the most used database libraries.
Anyways, my point is that category theory is not necessary nor necessarily useful to use scala, including it's functional abstraction libraries.
Clojure sounds nice. I haven't worked with a big system lisp, I've only done small functions and such. I've heard writing code in a Lisp is a pleasurable experience.
>>18154So what do you propose?
There's some interesting imageboard concepts. There's a 3D imageboard/forum space. It's pretty weird. boorus are also considered imageboards. Reddit-style, or digg style forums aren't really imageboards. "bb" forums I guess are imageboards, assuming image posting is enabled, and are also the OG forums after usenet and mailing lists. I want to think that the format of imageboards and BB forums works for what they are used for. Namely sequential discussion. And reddit-style/HN-style forums are good for mainly one-off comments surrounding a specific OP.
I wonder if there's a way to get the best "features" of both platforms, and come up with a unique forum experience. How would this look?
>>18795 (me)
Besides reddit, another very popular platform is fbi.gov. As a non-discord user, I tend to underestimate the importance of the platform.
A good community nowadays has way less expectations of privacy than the internet in the early 00s and 10s. Interactivity, chat, quick response, self-moderation, hierarchy of moderation (drama), drama, lightweight "channels", etc.
One complaint I read about old BB style forums is that they're hard to parse. Part of it is due to the shit reply system. Another part is due to the layout of blocks per post, whereas fbi.gov has a more compressed view. There's also the ability to "spam" comments. On imageboards, each post is a huge block.
>>18796Thanks. I'll definitely check it out. Might contribute even. My goal is mostly to learn Go, not to build anything. Unfortunately I don't have much time to spare to learning anyways.
>>19091I don't see a reason to use PHP in today's world. All other languages are good enough. Everyone's learning other languages nowadays.
PHP is not so different to, say, python or java. Just use what you're comfortable with and bias less shitty languages.
Btw, there was a work in progress for a technical design document for a new imageboard, no? I keep revisiting the idea of wanting to build an imageboard as a hobby, but it's way too overwhelming unless I have some reference to program from. I'm thinking of writing a TDD, but the amount of effort that would take is way more than the time I actually have for hobbies.
I'd start with a list of features, grouped by type, and separated into two or three phases. Phase 1 would be "mvp". Phase 2 would be "feature parity with vichan". Phase 3 would be "nice to have".
The description of Phase 1 and Phase 2 features would be the most important.
Afterwards, I'd start adding API endpoints to the features. How exactly will the front end achieve these tasks. I'm assuming the system will have an API. It's also important to remember the requirements, if the site is available with no javascript, then the API must account for that.
Then I'd start sketching out request and response objects for the API. Nothing too fancy.
Afterwards, I'd start building entity relational models based on the APIs, the constraints, and the request and response objects.
And I think the last step would be to annotate each feature with a description of the Front End implementation. How exactly will it be implemented in the front end?
So at the end you'd have a document that says something like:
Phase 1:
Posting:
User can create a thread:
The user can create a multimedia thread. The OP has name, subject, etc.
API:
POST /threads
or maybe
POST /boards/{boardId}/threads
Request object:
name: Text
body: Text
Response:
id: uuid
Related DB storage:
thread:
- original_post: Post (fk post) <- this is required for the implementation because x and y
- created_at: date
- is_anchored: bool
(this db part is just an example, doesn't have to be this way, it's loosely based on the example here
>>14251)
Administration:
A moderator can do X or Y:
etc etc…..
The point of doing this would be to make sure that all the features of vichan are accounted for. This would be language, framework, etc etc agnostic. Perhaps the only non-agnostic part of it is that it probably assumes a relational DB. The other important part is that it lays down all the features and forces them to play together, allows them to be inspected and questioned. It will essentially force you to build the software taking into account all the features that are expected of the system.
Hopefully this document would serve as a guideline for the future developers who decide to build an imageboard system.
>>11528>You should insert the files in the proper order and then order by id. there's no need for a "file order" field.(not rat)
I disagree actually. Sequential ids are no longer "in". UUIDs are the future, kid. So file order is relevant here.
>>11265Damn was I thorough. The amount of shit going on is just insane.
I found a federated imageboard software, and apparently the only remaining federated imageboard
https://usagi.reisen/ I have no idea how ActivityPub works, but maybe it's a good thing to have at some point to ensure … something? Not even sure how none of this works.
>>21996PHP is easy and a lot of people know it. You'll get many more people working on it. If you keep it to some niche language or framework like OCaml or Laravel, then you'd be best going to those language's communities and seeing if there is interest there.
PHP is a decent language for imageboards. Hell, you can even write a backend server in PHP to handle the requests and the database, and the API.
>>22012Basically all procedural languages are pretty much the same. My issue with PHP isn't that the syntax is hard to learn or whatever. Basically all procedural and popular languages are picked up very quickly as long as you know another.
My issue with PHP is that it takes a lot of knowledge to not shoot yourself in the foot doing normal stuff. My issue with java is that it's a worse kotlin. Doing basic stuff you're probably going to be fine, but you'll also be fighting the libraries to find what you need and figuring out what you need to do to instantiate a SimpleObjectWriterFactoryInstance<ObjectWriterImpl>. Might as well just use kotlin, which comes with much better type protection than PHP or java.
Kotlin is an improved PHP. If you know PHP, you can do kotlin.
>>22013Why do you say so?
>>22015It was a jab at java, not PHP.
https://stackoverflow.com/questions/2885173/how-do-i-create-a-file-and-write-to-itLook at the hundreds of different ways to do something very simple: write shit to a file.
Half of these solutions will give you magic runtime errors if you try to run them.
>>22112personal preference of CL over scheme and also because as I said before, an imageboard is such a trivially simple piece of software that it's not like it would be unreasonably difficult to make another one, as much as it also is kind of boring and pointless
I've seen this textboard engine though and think it's neat for what it is
Just skimmed some of this thread, but looks largely like a design by committee selecting signs without anyone to actually move the software part of the project (except for maybe some work on the schema). Ages ago had a thread on this, one of the only active threads on /tech/ making an imageboard in OCaml. Ultimately the front-end was completed (with mobile site) to that but only the parser of the backend ended up in OCaml. Most the rest of a backend (in Java) but never really tested any of it because the machine it was written on didn't have a functioning (Java) installation at the time. (so is likely completely non-operational)
Not
>>22102 but just made some good progress on my own textboard written in CL (with a few lines of CSS taken from SchemeBBS). It has login, posting, bans by user, and address; but is simpler than typical in that the thread view just shows the thread, and threads don't bump, and messages aren't parsed just escaped. Thought about doing something sophisticated with tags instead of boards like #Tech, but this is a bit more than have the energy for at the moment. Just wanted to post to prove that you can implement a simple model in a matter of days.
Might try to skim the thread a little more to see if there's anything interesting.
>>25933haha. (it was nice to use though) There was a lot of this in addition to
>>25932.
i cant program or anything like that but sometimes i imagine myself creating an imageboard (I'll never do it) and I have a very solid idea that I daydream about
the boards are
>hallucinogenic drugs
people posting trip reports and shit
>dreams/surreal
people talkibg about dreams and teh way dreams work (I love dreams)( anything dreams really
>philosophy and social sciences
self explanatory
>OC
board in which people post OC and talk about making OC, includes music, memes, drawing, writing, poetry and whatnot
>art consumption
board in which people speak of what shows theyve been watching or what music they listen to or like anime or whatever
>powerless
board for people who are lgbt, neurodivergent, black, poor or whatever other thing that is disadvantaged in society to speak about their unique struggles
like and subscribe, oh and also every board has its own specific board tan who is a cute girl and embodies the vibe the board is supposed to have and the whole thing has a dreamy purple color scheme and the artstyle is very reminscent of tkminz (idk how to write, the girls last tour person)
I think it would be very nice
>>25931I have written two "fully featured" imageboards; the first in c++ and the second in one of those "better c" languages. imageboard logic maps comfortably to db features like triggers, prepared statements, views and so on, you can write most of the logic as sql queries. personally I don't like it, but you could go as far as storing the site images and thumbnails as binary blobs, or have the server config be a table instead of a file. bumping threads for example can be a sql trigger, it might even be more efficient than querying the database multiple times (add the reply, bump the thread, commit)
the tradeoff is that the sql syntax for these things sometimes depends on the dialect, so if you use, say, mariadb syntax, your ib will be locked to mariadb. ime, for this particular use-case, orms add more boilerplate and overhead than just using a db driver (or a thin wrapper around it)
tinyib templates are easy to
steal understand if you don't have the patience to write html and css
>>25933>>25931You know that if we want to do a new imageboard, we should use something that is used by people?
No shit there is no progress if everyone picks their own niche FP language with 10 users and no advantage to more used languages asides from "muh purity", "muh syntax" or some other childish thing.
If we want to get something done we have to pick something familiar to people. I do backends in C++ Drogon when they are small scale and only i will work on them. Even though in my opinion it is a very qualitative framework, i do not do backends in it when i expect others to work on it since they don't know it. But the FP fan's ego is too large…
Not a lot of people will want to contribute. Add into that that they need to know your obscure FP language and it's over.
If we want to get something done, people that want to contribute should hop into a matrix or slack, and pick the language based on who knows what and who would work the most.
>>25940You cannot learn OCaml in an evening without strong prerequisites. Even more impossible when you consider "learning" being able to do a large scale codebase with it.
With C++ you can do most without knowing a lot of features.
This is not a languewar thread. Every unknowledgable person can languagewar for years. You kinda prove what i've said, if we argue over which niche technology to use (at least niche in web backends), we will get nowhere.
Let all those that want to act instead of blabble mindlessly and aimlessly organize, and only then we will pick the stack. But everytime a "let's cooperate to do X" project gets to the actually doing stuff part on this forum, everyone ghosts.
>>25941>With C++ you can do most without knowing a lot of features.lol. this is specially funny considering you want to write something that will be exposed to the internet
>say something dumb>get called out>no! you are derailing the thread!this thread had been dead for 6 months just 120 minutes ago, and you only interceded to trash talk a guy posting his project. what an organizer
>>25942>lol. this is specially funny considering you want to write something that will be exposed to the internetDid you read what i said? I said that even though i like C++ Drogon not a lot of people like/know, so i recognize that we shouldn't use it and it's the same for OCaml.
>this thread had been dead for 6 months just 120 minutes ago, and you only interceded to trash talk a guy posting his project. what an organizerI saw it high up because some other person posted, i did not trash talk all i am saying is that OCaml might be too niche
>>25947Yes but do you need to learn the intricacies of aliasing in C++ for any program? And i do not know if you mean using or typedef but these are the most obvious keywords.
The point would not be to do a nu-imageboard with unknowledgable people but with knowledgable people. A language is easy to learn, software engineering is not, and if the codebase is made by people that can't software engineer it's over.
Anyways, i might get this going somewhere. I have a bunch of colleagues with whom we wanted to do an app as a side project but didn't really know what. I have proposed them the idea of a more modern imageboard, and i am waiting for their answers.
>>25937Hadn't heard of SQL triggers before, that's an interesting approach. Would be interesting if it was in the ORM familiar to me (don't particularly like maintaining multiple representations but it seems nearly necessary).
>>25936Was art therapy looking back. The current one is 300 lines and half of that is SQL, queries, or structs to match the database (the rest is mostly HTML template). Anyway ended up writing most of the original larger imageboard in (not running) Java (except for the rather nice front-end obviously). Not dogmatic about functional programming, and not sure CL is a "functional programming language" or not. Seems some are missing the forest for the trees.
Picking tooling plausible contributes (e.g. a vote of individuals with repos and sign a paper to commit some support) are comfortable working in is certainly a reasonable thing to do if one was interested in gathering support.
>>25931>.. selecting signs without anyone to actually move the software part of the project (except for maybe some work on the schema).Am sorry to have written this, it really wasn't entirely reflective of the thread but just some bad trend.
Probably going to go back to not writing here, though it wasn't seemingly too harmful to visit again. >>25949even if you write multiple representations, you still lose things like automated migrations (if the orm manages that). triggers do nonetheless save time on the small stuff; you can add admin logs, for example, with just a few triggers
another interesting topic related to the schema is normalization vs optimization. even in something as simple as an imageboard you have that a highly normalized (above bcnf) schema can be more readable and correct, as there wouldn't be as many opportunities to represent invalid states, but it can also be significantly slower (require more joins). and it might depend on how the database implements views
>>25948aliasing as in the "__restrict" keyword
>>25951The larger version of the ORM am using has automated migrations and schema versioning, apparently. Though this is not what was being naively referred to by me.
Regarding the schema my plan was always to just have one main table,
posts (along with
users and
addresses), looking at your previous comments, this appears to be how Lainchan does things. Conceptually speaking however threads aren't in anyway different from boards or posts in my program. A board is treated identically to a post with
(= id 0). Each level of indirection appears to hide replies, and can go on ad-infinitum with posts treated as new threads or boards for replies.
Implemented markdown parsing this morning just for fun. If it were styled correctly would consider it a reasonably complete program.
>>25956Write it in assembly
>>25948>Anyways, i might get this going somewhere. I have a bunch of colleagues with whom we wanted to do an app as a side project but didn't really know what. I have proposed them the idea of a more modern imageboard, and i am waiting for their answers.Update: We'll have a meeting, some are favourable, others are not, i will update you guys.
>>25964>Decided this was a bad idea; it now displays with boards and threads.Bad idea.
I'm thinking of two DB models depending on how we want to do it. No matter what we use, we will use ScyllaDB.
If we want something decentralized, where one server = one board (like every imageboard right now), we have multiple sub-boards each with their own name and id (/tech/, /AKM/ …). We also unite the concept of thread reply and thread, with just one bool or special ID or something to indicate if it's a thread or not.
If we want something more centralized (kinda like a 'ddit of imageboards), we have multiple "boards" which can hold multiple "subboards".
Main idea is uniting the thread and thread reply.
>>25976Edits:
by "bad idea" i mean that you're right that it's a bad idea and you should keep doing what you're doing.
And in the "centralized" idea we unite thread and thread reply. We are software/embedded devs so the model might not be the best for scalability.
>>25976I decided to check scylla/cassandra out the other day when someone mentioned it. I made a small imageboard which I might post today or tomorrow in c++/drogon/scylla. I don't think any imageboard needs the scalability, the entry cost is relatively high (you need a partition with XFS to get the advertised performance, for example) and it is hard to get reproducible builds. cql does however force you into efficient table schemas, and the c/c++ driver is well documented and easy to use in general
the main problem is that drogon (and similar c++ libraries) uses an event loop and lots of global state, you can use coroutines or callbacks to not block the thread with the cql queries, but the drogon code is hard to read so I'm not 100% sure it doesn't screw with the event loop. I don't know if this means you are supposed to integrate the driver into the drogon orm
another problem is that c++ in general has bad meta-programming, writing the serialization code manually with the cass_value_get_ primitives takes less time than writing the macro/template hell you could write to automate the process, languages like nim, zig or rust don't have this problem
and drogon uses exceptions, this means you have to use RAII, the scylla driver does not, you have to either write wrappers for the entire driver, or use the unique_ptr destructor field. the problem with wrapping everything is that you need a very good understanding of c++ copy and move semantics or you (or someone using your wrapper) is most certainly going to make a mistake somewhere. using unique_ptr is easier but it clutters the code and kills readability, for example, this:
auto* statement = cass_statement_new(query, 0);
becomes this
auto statement = std::unique_ptr<CassStatement, decltype(&cass_statement_free)>(cass_statement_new(query, 0), &cass_statement_free);
I hate c++ and I hate myself for taking the bait and trying to do something with it. this was a deeply alienating and tedious experience, I can see why people get paid to do this. anyways, I think it would be better if you took advantage from the fact that the driver has a C interface and used it with rust or nim instead of c++
but even if you do go through with it, scylla requires so much setup, I just can't imagine someone choosing your hypothetical software to host their small alt-chan
>>25964>>25953>>25952>>25949>>25931Decided to migrate to a tiny webframework unfortunately named
ningle and now the better part of the project is broken without any easy way to go back since haven't used version control let alone continuous integration. May or may not try to fix it at some point.. Anyway probably done posting for a while, peace!
>>25981It'd be a bad horse (if that's what one is) of me to leave on a failure. Seem to have managed to get everything fixed and in working order. Even in 386 LOC it's a usable application. The primary things missing are: moderation UI (rather than using a REPL), file uploads (rather than embedding via markdown), traditional syntax (rather than markdown). Considering the login mechanics especially could see it being used for something like Ismail's /marx/;
though not in fact Ismail's /marx/.
Part of this is due to concision of CL
>>25981 is the entire system for managing user login and signup
other than the SQL, and routes. Wonder if there is a way to make a permission protected REPL to have moderation staff use without complete trust?
>>25976>where one server = one board (like every imageboard right now),Didn't realize this was the case. This means you can at least have different boards with different moderation staff without implementing UI for such a thing.
>>25980>you have to either write wrappers for the entire driver, or use the unique_ptr destructor field.This sounds very unenjoyable, don't suppose anyone has made a library of driver wrappers?
>>25984I haven't checked but I assume most people use them with libraries like seastar or libaio instead of drogon. no exceptions -> no need for raii -> no need for wrappers
ig accepting the occasional leak can be an option depending on the use-case
>>25980Sorry that you thought i wanted others to use this, Drogon and C++ is not for the new-ish user. Most of what you say sounds like the usual "Nim, Zig or Rust" user having a bad day, although i concede that until C++26 reflection, the ser/de related metaprogramming is bad. There's libraries similar to rust serde (in the sense of no ser/de boilerplate) though, such as glaze (its also stupidly fast).
Thanks for the feedback on Scylla though, i do not know shit about DB's, all i know is that i like performance XD.
Drogon has support for Postgres via an ORM or raw SQL, might want to check that out.
>>25986I have used drogon intermittently since 2019, what do you mean. it doesn't change the fact that most c++ code is write-only and the framework uses global state
and it can affect performance because you never really know what the library is actually doing. I will put it this way, I have never read documentation for a zig library (not even the standard library or the compiler) I just go and read the code. I can't say the same of c++
>>25987ideally? maybe. but if you have a time constraint you have to ask yourself if the boilerplate is really worth it. personally I like how you can use goto in c to emulate defer semantics:
x = resource_1();
if(x == error) goto d1;
y = resource_2();
if(y == error) goto d2;
//...
d2: free_2(y);
d1: free_1(x);
>>25988Well once again i don't want to try to argue, it's pointless, we all have our semi-justified preferences (including me).
What i like in zig is the compiletime stuff, i hope C++26 reflection will be as good!
Aight it's been delayed to tomorow.
Do you people think an app with the concept of an imageboard (threads, images, anonymous, boards…) with modern UI/UX, marketed towards normies could work?
It would fix concerns more and more normies are having about data collection, trace you leave on the internet, algorithms being bad…
POLL TO PICK LANGUAGES FOR THE BACKEND OF NU-BOARD
https://strawpoll.com/PKglezRdRZp>>11288>Would anons even want a "modern" style HTML5/CSS UI with flex/grid/etc. ?You don't necessarily need these to make responsive layouts, but you can use modern CSS to implement many features that used to require javascript such as menus and forms that hide on click, images that become large on click etc. This would probably appeal to many users.
>>26021>modern UI/UXPics related were the mock up front-end (have part of it as template code somewhere)
which had the OCaml backend. This is in the style of "progressing but not conforming" as mentioned by
>>11311.
Have never tried to participate in a group programming project before, so not sure where that leaves me on contributing.
I shit these out two days ago, I'm rather disgusted by it. this is how I would write the backend with scylla or regular sql if I had to do it in c++. notice that with c++20 you can use coroutines to avoid butchering the code into series of callbacks. moving the blocking cass calls to individual coroutines and then wrapping the c callbacks would be a good next step (something like this "SqlAwaiter" struct
https://github.com/drogonframework/drogon/blob/master/orm_lib/inc/drogon/orm/DbClient.h#L49)
https://git.leftypol.org/c_hater/dsqldhttps://git.leftypol.org/c_hater/dscd>>26021I will make the logoI don't think polls work for starting collaborative projects. maybe set a deadline (a week?) and ask people interested to write a "base" in their preferred toolchain; then you vote with something concrete. that is, if there were more than 3 people in this thread
>>26032since threads and replies use the same table (a thread is a row r such that r.thread = r.post) you need the bump field to sort threads by bump order. timestamp is for the time the post was created. in sql you could have a trigger like
create trigger bump_thread
after insert on post
begin
update post
set bump = new.post
where post = new.thread
end;
now that I think about it, in scylla/cassandra, the timestamp should be removed and the post column (the post number) should be replaced with a timeuuid (same with the "thread" and "bump" columns) because it doesn't have transactions. in the regular sql imageboard, the "board" table usually has a "post counter", so that post numbers are board-dependent and contiguous. adding a post usually looks something like this (pseudo code)
db.beginTransaction();
board = db.getBoard(board_name);
post.number = board.counter;
board.counter += 1;
db.update(board);
db.commitTransaction();
db.insert(post);
you use a transaction because the board.counter "fetch-and-add" has to be atomic, or you might end up with a collision (two posts in the same board with the same post number). but you don't have transactions in the scylladb api. the cql schema without that race condition would have to use timeuuid instead of post numbers. timeuuid are identifiers that contain a timestamp + a random seed
it will however change the user experience a little. instead of post numbers, posts will have to be presented to the user with either their uuid or a fragment of it. for example, the last 8 characters from the hex representation of the uuid. notice that, since scylladb doesn't support triggers, bumping threads still has a small race condition. fortunately, it does support something called "lightweight transactions" which do solve the thread bump race (but not the board counter one)
>>26021>Aight it's been delayed to tomorow.Today is tomorrow, here is the bits of notes i can share with you guys (pasted in Deepl, we're not anglos)
As the poll had not much results, the choice of backend stack is to our discretion. We're thinking of C++ / Drogon / Scylla.
I'm gonna start working on the backend as i have nothing to do.
Fun fact: I showed the 'arty to the boss as an example of an imageboard, he saw a 'cado.
Modern imageboard
- Make an imageboard with a more modern, ergonomic design
- Market to normal people as a social network that addresses the issues of data collection, Internet footprints, censorship and algorithms.
- Market to imageboard users as a more modern, fast and accessible backend and frontend than their current crap (vichan…)
- The goal is above all to get new users, but it's also possible to migrate an imageboard to our more modern one.
- The design is basically centralized, we let people create their own communities (ex: leftypol/soyjak.party/4chan) with their own boards (AKM/soy/qa) on our backend
- But we let people host their own backend for their community(ies) by making our backend available to them as open source.
- We let people make their own frontend like fbi.gov, by making their own app/website, ours (open source and by default) is a website.
- On our website, we'll have to moderate the most disgusting things (cp), but maintain freedom of expression.
- We could consider adding functionalities to a classic imageboard (chat, voice chat … ?)
Translated with DeepL.com (free version)
>>26035>since threads and replies use the same table (a thread is a row r such that r.thread = r.post) you need the bump field to sort threads by bump order. timestamp is for the time the post was created. in sql you could have a trigger likeWas thinking of something probably more convoluted,
also tried r.thread = r.post before switching to r.thread = 0 in my toy. Did recall that you could use triggers in this case, having learned of them from this thread.
>it will however change the user experience a little. instead of post numbers, posts will have to be presented to the user with either their uuid or a fragment of it.That is sort of neat.
Am curious how modern.
>>26060Just intuition, it's effective in some cases.
>>26061haha.
>>26062Leaving it to web-server level seems it would save some time. Might remove the caching..
>>26076
>in the team
What team? The staff team? They literally have an optional account lol. What was their argument for them?
It's important to distinguish types of imageboards. They're not homogeneous, and anyone who used older imageboards or certain outer-IBs knows that namefagging can be absolutely normal, especially on boards where people actually do things apart from edgeposting.
Chan imageboards are bascially just a special type of forum, and 'anonymous imageboards' are a common type of chan board. So while I do like anon culture, I disagree that it's "against the purpose of an imageboard", unless you're talking about a specific imageboard whose purpose necessitates anonymity.
>but it could allow an easier management of the moderation team of an imageboard
I don't see how it would. The ultimate problem is extremely low barrier of entry. These places are extremely easy to maliciously post on, despite some of the antispam measures. Forums with accounts take additional measures, like needing to verify accounts, or revealing forum categories gradually so you need dozens of posts/likes/etc. to get to the real content. Obviously that wouldn't fly on an imageboard.
As a side note, some boards do have optional user accounts awarded by the admin, like plus4chan has a VIP account for long-term donation support that gives a colored username.
>>26098>What team? The staff team? They literally have an optional account lol. What was their argument for them?The team i'm making the imageboard with
>>26099>His imaginary team, he is role-playingI'm a level 15 Arcane C++ mage
>>26134I prefer the second because you can then the query the entire thread with just
select * from simple where thread = ?;
a query you can then optimize with an index
create index simple_index on simple(thread);
>>26143maybe remove the semicolons at the end?
>>26144what do you think he is writing?
>>26148I mean, when you do (execute-non-query db <query string>), if it is a single query (a single trigger) maybe remove the semicolons at the end of the string
I know this sounds dumb but if you have multiple queries in the same string, see if you can break that string into individual queries
I'm reading
https://repo.or.cz/cl-sqlite.git/blob/HEAD:/sqlite-ffi.lisp and it seems like you could write a short binding to call the c function made for this stuff (
https://www.sqlite.org/c3ref/exec.html) from cl
>>26145add "for each row" to the triggers? and the language should go after the closing $$
>>26152Yesql
https://github.com/krisajenkins/yesql seems pretty ideal by contrast. Just write SQL in an SQL file queries return a hashtable keyed to the query. Almost makes me want to give Clojure a go, except for actually quite like the implementation of CLACK, LACK, and the tiny framework built on top; not sure if there's something that ranges from sinatra style routes to hand written HTTP responses like that.
>>26158honestly this sounds like a tooling problem. if you just want to be done with it maybe consider "adopting" the html templates from an existing codebase (
https://desuarchive.org/g/thread/101401270/#101406085)
>>>/tech/26160repo or bait
>>26162Technically already have something like templates implemented, used to render:
>>26023 though would want to rework the catalog to look more like
https://lobste.rs or
https://news.ycombinator.com where you can easily search for older threads as mentioned here: >>>/siberia/557113
>>26163Also think the catalog view
should matter dramatically more than the typical board view to the point where you could get rid of it.
>>26190>surely the millionth vichan clone will make hundreds leave social media and flock to imageboards this timeMarketed towards normies it could know success. Let me believe.
Btw according to the designer his design is almost ready, then he will have to do the frontend
>>26199>Marketed towards normies it could know success. Let me believe.Imagine it would have to deviate rather markedly from the format to do this. My routes are broken, and considering abandoning my project, but imagine a live imageboard's algorithms, and anonymous posting with
https://lobste.rs style accounts, invites, tags, pagination, and catalog page. Could be a real winner.
Would be up to moderation and the tagging mechanism to keep it from being disgusting.
>>26247>Imagine it would have to deviate rather markedly from the format to do thisYes, we're going for a radically different look. The current iterations are very far from the norm in terms of imageboard.
>My routes are broken, and considering abandoning my projectOh, what's happening, you''re already far in your projet! Ours might fail, we're not web devs, don't let ours discourage you!
>>26249>Ha, nothing much; suppose that's the main thing. Enjoyed writing the SQL for the most part and some of the business logic but not as fond of the template or making the end points. Haven't gotten to the point of appreciating a little bulk…Maybe not enjoying writing the endpoints is due to the stack you use? We all have our preferences, and some stuff can be more verbose/hard/unfunny than other
>>26250>Might also be good to just "non-compete" out of computer science.How? Don't drop out of something you can like just because of that!
>>26266>Aight, our designer is having some issues with seeing what could a modernized and ergonomic imageboard could look likePlay around with
https://nuclearchange.net/social/That shit's shnazzy yo. Definitely a step-up from here with the real-time replies and overall smoothness, and some of themes are more modern (in the settings menu).
There's also lainchan's optional keyboard controls and WYSIWYG, I haven't used those but it's ergo.
>>26328the normal way of doing this is a (pseudo) radix tree, if you feel like reinventing the wheel. that way you can have regex routes and such without hard-coupling. it is the type of thing you can bury below layers of abstractions
ultimately the only thing the program does is shuffle data between the http requests, the database and the html templates. in a high-level language like lisp I imagine there must be ways to abstract this to a point where you can add new endpoints just by describing what comes in and what comes out
>>26400geg, maybe 5chan. However, the fact it's written in C++ is just a detail, this wont be another gochan or jschan.
>>26036 is our goal, we still have to make it clearer though.
(at least we hope so)
Unique IPs: 79