/tech/ - Technology

Non-SPA:
-loads faster client-side
-works better with tabs, back/forward button, scroll placement etc (at least without a lot of extra accommodation from SPAs)
-works better with search engines
- some other stuff I'm forgetting rn?

SPA:
-smaller data requests for new data without refreshing page
-can do some complex shit client-side
- also other stuff

>>6622
which one would be better for an imageboard?

>>6620
>>6623
Applies to basically any type of site. IMO client-server is better than only server side.

-Swappable UIs (if the UI sucks for whatever reason, you can re-do it completely)
-Same back end for different services (web app, mobile app, etc)
-Less server load and smaller payloads.
-Separation of concerns make the code easier to reason about.

Some bad things include:
-Worse initial load
-Worse SEO
-Requires javascript
-JS libraries are absurdly bloated.

>>6624
isn't the downside also that angular/react get updated like every 5 months so you'd have to keep it maintained? well that and the paranoid /g/entoo-fags who disable all brower javascript wont be able to use it.

If theres a headless imageboard it would be nice

>>6625
Javascript land is insanely insecure and a fucking mess. As long as you keep your dependency graph relatively small and shallow, you should be fine not updating your libraries in a while.

There are also other solutions, like using a language that compiles to JS. If on top of that you use a language that is strongly typed, you probably won't have many security problems and will probably be fine staying with whatever version you have. And if you ever update, the compiler can help you out.

There are downsides to that as well, it's not a silver bullet.

I'm actually very slowly writing an API for an imageboard. I'm using it to learn, mostly. If I do it right, I guess I'll publish it. To get feature parity is pretty hard. For example, post.php of lainchan is 1530 lines of code. It's a fucking mess and can probably be written in half of those lines, but that is just one of many many files.

To give an idea, this is lainchan:
17 403.php
34 404.php
15 banned.php
15 banners.php
20 bg.php
20 c.php
1006 install.php
24 log.php
216 mod.php
28 player.php
1530 post.php
19 report.php
174 search.php
86 smart_build.php
54 staffapplication.php
298 inc/anti-bot.php
209 inc/api.php
320 inc/bans.php
173 inc/cache.php
1897 inc/config.php
108 inc/controller.php
165 inc/database.php
451 inc/display.php
91 inc/error.php
45 inc/events.php
253 inc/filters.php
2889 inc/functions.php
502 inc/image.php
366 inc/instance-config.php
39 inc/lock.php
187 inc/polyfill.php
49 inc/queue.php
64 inc/remote.php
65 inc/route.php
78 inc/template.php
53 templates/themes/basic/info.php
40 templates/themes/basic/theme.php
57 templates/themes/calendar/calendarpost.php
36 templates/themes/calendar/info.php
40 templates/themes/calendar/theme.php
110 templates/themes/catalog/info.php
506 templates/themes/catalog/theme.php
42 templates/themes/categories-uboachan/info.php
223 templates/themes/categories-uboachan/theme.php
67 templates/themes/categories/info.php
88 templates/themes/categories/theme.php
81 templates/themes/donate/info.php
28 templates/themes/donate/theme.php
36 templates/themes/faq/info.php
33 templates/themes/faq/theme.php
55 templates/themes/frameset/info.php
61 templates/themes/frameset/theme.php
38 templates/themes/irc/info.php
27 templates/themes/irc/theme.php
33 templates/themes/public_banlist/info.php
57 templates/themes/public_banlist/theme.php
58 templates/themes/radio/info.php
27 templates/themes/radio/theme.php
54 templates/themes/rand/info.php
105 templates/themes/rand/theme.php
88 templates/themes/recent/info.php
167 templates/themes/recent/theme.php
75 templates/themes/rss/info.php
198 templates/themes/rss/theme.php
22 templates/themes/rules/info.php
27 templates/themes/rules/theme.php
70 templates/themes/semirand/info.php
257 templates/themes/semirand/theme.php
62 templates/themes/sitemap/info.php
48 templates/themes/sitemap/theme.php
36 templates/themes/staffapplication/info.php
54 templates/themes/staffapplication/staffapplicationpost.php
40 templates/themes/staffapplication/theme.php
99 templates/themes/stream/info.php
34 templates/themes/stream/theme.php
23 templates/themes/zine/info.php
28 templates/themes/zine/theme.php
14790 total
That's 14k lines of code. A lot of these are not used at all or are config shit. The codebase has a huge issue of dead code.
Perhaps a more "fair" assessment might be this:
17 403.php
34 404.php
15 banned.php
15 banners.php
20 bg.php
20 c.php
1006 install.php
24 log.php
216 mod.php
28 player.php
1530 post.php
19 report.php
174 search.php
86 smart_build.php
54 staffapplication.php
298 inc/anti-bot.php
209 inc/api.php
320 inc/bans.php
173 inc/cache.php
108 inc/controller.php
165 inc/database.php
91 inc/error.php
45 inc/events.php
253 inc/filters.php
2889 inc/functions.php
502 inc/image.php
366 inc/instance-config.php
39 inc/lock.php
187 inc/polyfill.php
49 inc/queue.php
64 inc/remote.php
65 inc/route.php
78 inc/template.php
9159 total
Say you can write this if you have experience and acumen in 4.5k lines of code. I've heard that people write 20 lines of code a day on shitty desk jobs (https://skeptics.stackexchange.com/questions/17224/do-professional-software-developers-write-an-average-of-10-lines-of-code-per-day). If that's true, this would take a full year of work. Something more reasonable, since you already know what your building and this is more personal work is to write maybe 40 lines of code on average during the whole endeavor. That's around half a year of dev hours. These things are really hard to estimate, and they're only back of envelope estimations based on a billion assumptions. There's also so much dead code and now-unnecessary workarounds.

The front end has:
Front page
Index view
Catalog view
Overboard
Overboard catalog
News
Logs
And around 6 shoddily done mod pages
It's rather simple for a front page.

>>6626
I've searched for a lot of headless imageboards and I haven't found anything decent.

>>6627
>I've searched for a lot of headless imageboards and I haven't found anything decent.
yeah i mean writing one
>Say you can write this if you have experience and acumen in 4.5k lines of code.
supreme reality had < 1k lines of code, OFC it was written in a functional language. probably like 400 lines of code if you don't count database queries.

>I'm actually very slowly writing an API for an imageboard
nice

>>6628
Where's the code?

>>6629
https://github.com/8b2eef7c901269e8e9a6ec532d57b6b1/supremereality

knock yourself out, doesn't look maintained though

just to clarify thats not headless, its server side rendered HTML

>>6630
>>6631
Awesome. Looks really cool.
Do you have any idea on what your projects is going to use?

This is the lainchan databse btw, for reference. As you can see, it's relatively simple.

>>6632
I'm still in the architecting phase. I'm leaning towards java though (likely spring / spring boot), being that its a popular enough language so that most people can understand it, but at the same time its typed. I know a lot of anons here really want to use a typed language

>>6633
I like stronger typed languages, but I also like python. Django is a joy to use for small projects.

>>6623
>>6633
>I'm leaning towards java though (likely spring / spring boot), being that its a popular enough language so that most people can understand it, but at the same time its typed. I know a lot of anons here really want to use a typed language
if static typing is all you want, it would take much less time to just fork lynxchan and rewrite it in typescript. rename your fork tynxchan or something. if you don't like the disorganized mess of mongodb, swap it for postgresql.

don't make the same mistake every overambitious programmer makes, ie., a phenomenon that has several terms to describe it: law of triviality, bike-shedding effect, reinventing the wheel, not invented here syndrome, etc.

>>6627
>If on top of that you use a language that is strongly typed, you probably won't have many security problems
famous last words

>>6661
>don't make the same mistake every overambitious programmer makes, ie., a phenomenon that has several terms to describe it: law of triviality, bike-shedding effect, reinventing the wheel, not invented here syndrome, etc.

That doesn't really apply because rewriting lynxchan in typescript to use postgres would be rewriting like 80% of the software, you might as well just make new software. I've worked on imageboards before (not specifically lynxchan) and at least half the code is typically the database queries. To a certain extent an imageboard is just a glorified CRUD app

Anyway IMO Java is a better language than typescript, which IMO was just invented as a cope for corporate programmers to be able to have types on JS.

>famous last words

I agree static typing doesn't solve most security problems. However static typing does increase runspeed because you don't have to check types at runtime. Also I just like it better than JS/TS because im used to using it at work

>>6663
>you might as well just make new software
if you're going to start from scratch, use rust w/ actix-web and sqlx

>However static typing does increase runspeed because you don't have to check types at runtime.
rust is even more performant

https://www.techempower.com/benchmarks/
>actix-core (rust): 651,144 req/s
>vertx-postgres (java): 347,356 req/s
>spring (java): 27,339 req/s

https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/revcomp.html
>Rust: 0.46 s, 499,024 b
>Java: 1.54 s, 702,332 b

https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/regexredux.html
>Rust: 0.78 s, 146,236 b
>Java: 5.70 s, 656,328 b

>>6675
>if you're going to start from scratch, use rust w/ actix-web and sqlx
Rust is based from what I hear but I also don't know it. Maybe you can make a Rust thread and post a pdf or something.

I'm aware that Rust can be faster than Java, I just don't know it well enough to write a massive project in it

>>6675
server side,
There is no real reason to care about performance nowadays in almost all scenarios. Almost all server software runs sufficiently fast and language/frameworks will not be the bottleneck.

Use whatever you like. Prioritize comfort over performance.
t. I do this for a living.

>>6826
>server side
I agree the only problem is my CSS is weak, id probably want someone else to help with styling/theming the board

>>6827
lel. I wrote that you don't need to worry about speed, then I wanted to be specific that I meant specifically on the back end.

That's true regardless of whether you use server-client or server-side rendering.

server side vs SPA is that some people don't use javascript. There are solutions where you can do server side rendering and if the user has javascript, the SPA kicks in. Havne't used it myself, can't vouch for such tech.

>>6828
I'm very tired sorry for writing like a WSWS on their 10th article of the day.

>>6829
>>6828
ok. My ideal solution would be both. A server side rendered version of the app, but also a restful interface on a different route that anyone could write a client against.

So if they go to / then they get the server version but all the api endpoints are on /api/ or /apistd/ or something

the only reason I would want that is because i want existing imageboards to adopt the standard which shouldn't conflict with existing routes they use for other shit

>>6662 : related, developing some standard imageboard API which allows people to just write their own servers/clients/etc. which is more valuable than even just writing yet another imageboard

>>6833
There already exists a standard api of sorts. The 4chan api is standard I believe. This site has a "standard" api of sorts. Makes sense to at least offer that API so that existing apps can be ported easily.
>>6831
definitely possible. it sounds easy in my head, but I haven't done it myself.
if you make a normal server-side-rendered site that also loads a JS that's a SPA that hijacks the entire dom when loaded, then that should work relatively easy. The SPA would then use the /api/ routes if you'd like.
There are stuff that allows you to render the SPA as if it were a static page. I've read mixed reviews. It helps with SEO and with "time to paint" so that the user doesn't have to wait for the JS to load then for the SPA to download the data for the site, then to process, then finally to render.

>>6834
>The 4chan api is standard I believe
yeah but the 4chan api is read only AFAIK. This would also allow people to post and create threads

>normal server-side-rendered site that also loads a JS that's a SPA that hijacks the entire dom when loaded, then that should work relatively easy. The SPA would then use the /api/ routes if you'd like

I'd probably make the server side rendered version use traditional web app methods with GET/POST so the no-js/stallmanite/gentoofags would be happy with their darknet level, absolutely no js thing.

Maybe im just overthinking this shit and i should just make a restful/headless imageboard and let people write their own clients, maybe even a desktop app in the style of discord. I feel like id have to provide at least a crappy initial SPA frontend though, at least for demo purposes, though people could improve upon it.

IMO the most important thing would be to make that API, possibly by extending 4chan's api with some routes for making new posts/threads, reporting, authenticated actions etc.

>>6834
My plan is to not give a fuck about APIs and just roll out your own for now. In most HTTP frameworks, that is trivial to change.

For now, this the general API I've come up with. Each endpoint basically assumes CRUD (create read update delete). This is based on the functionality of this software, which is the most complete of I know. My idea is to reach feature parity first, then expand functionality. This is merely a learning experience for me, so I don't think I'll end up finishing it. I'm passing on my notes.

Authed endpoints, all of them are prefixed with: auth/

Auth:
login/
logout/

Manage Users:
users/
users/{id}

Mod logs:
logs/

News:
news/
news/{id}

Board settings:
boards/
boards/{id}

Reports:
reports/
reports/{id}
reports/dismiss/all
reports/dismiss/IP/{ip}

IP posts:
IP/{IP} (eg. delete by IP accepts board as query param)

Bans:
bans/

Posts:
posts/ (no create, also this endpoint is suspect, more below)

Post File:
post/{id}/files/{id}

Threads:
/board/{board id}/threads (also suspect)
/threads/merge/{source}/{destination}
threads/{id} (also suspect)

This assumes that posts, boards, and threads all use global IDs. Specific board >> ids would still exist. I haven't figured out how yet, but I figure that working with global IDs will be more comfortable.

Suspicious endpoints:
I still haven't decided how to supply mod information when a mod sees a thread. I'm thinking of just using the regular non authed endpoints and checking for auth info there. If the user is authed, and has proper mod permissions, then get the info. Or perhaps just mirror the non-authed endpoints and make them authed. I prefer the latter one.

eg:
how to see a thread:

boards/{board id}/threads/{thread id}

and if you want the same information but also with extra info for mods, instead call
authed/board/{board id}/threads/{thread id}


Then for non authed endpoints, you pretty much have:

boards/
boards/{board id}/threads
threads/{thread id}
threads/{thread id}/posts
posts/{post id}
news/
logs/


I'm still undecided on whether to make it like that or to nest the resources like proper REST.
eg. to edit a post

boards/{board id}/threads/{thread id}/post/{post id}
instead of just
post/{post}

But anyway, this should give you a rough idea on the resources that would need to exist for a semblance of back-end feature parity with vichan.

>>6838
I assume CRUD also relates to the normal POST/GET/PUT/PATCH/DELETE

>>6857
correct. I usually don't use PUT.
POST - Create
GET - Read
PATCH - Update
DELETE - This

How's it going, anon?

>>6879
Still working on it. I'm rethinking some imageboard concepts here.

Been busy with alot of real life stuff ATM pretty stressed out but ill be getting back to it shortly

>>6882
damn that looks based.

My reach is much more modest. I've been going very very slow, not by choice, but because it's been hard getting shit to work.

Right now I just finished the first 3 endpoints.
GET /boards
POST /boards (create)
GET /boards/{id}

Next on the list is some basic error handling. Then probably auth.
Hopefully stuff will move faster after that. If I had to guess, I'd say I'm somewhere between 5-8% of the way in terms of effort to reach parity with vichan. This is only the api/backend. Once auth is done I'll have a better idea.

I've decided that at least for now, it will be a highly opinionated backend, otherwise the matrix of configuration will explode the complexity.

>>6887
based. what language/framework you using?

>>6895
scala with http4s and doobie, using postgresql. I'm planning on long term, if I ever get to it, adding redis for caching.

>>6901
nice

>>6901
scala is too big brained for me tbh I had a hard time using it in school, but it seems pretty based

>>6906
With enough pain, most programming languages eventually become usable.
:')

We have like 20 tech users, so I realize the futility in reporting advances to my hobby project.

I took a break, but I'm back to working on it.
Error handling setup is mostly done. Now I just need to add errors whenever I make endpoints.

Auth was a pain in the ass. It's mostly done. Just need to write the logout routes. Everything is manual compared to ready-made auth endpoints from other frameworks I've used. That said, I am using an auth library that verifies JWT signatures and helps with authentication middleware, so it's not all bad.

Next on the list is adding more and more endpoints, which implies database changes, models, queries, error handling, etc. I decided I'd use a database modeler. That way the design is documented somehow. The program is called pgmodeler. It works really well. The pre-built binaries cost, but the code is open source. It wasn't too hard to build, luckily.

It has a "diff" function, so it checks my local database, and writes SQL to apply changes. This makes it easy to write migrations. Pic related.


I'm around 900 significant lines of code right now, which I calculate is around 1/10 of the way.
Just the handle_post() function of lainchan is 1k lines of code lmao.

>>6984
thanks for reporting in anon, other devs are interested in your work. are you the scala guy?
>pgmodeler
looks cool. I come from the oracle world so i had no idea this existed. Thanks
>I'm around 900 significant lines of code right now, which I calculate is around 1/10 of the way. Just the handle_post() function of lainchan is 1k lines of code lmao.
I have a feeling most of that is bloat, tbh its not surprising that with a more expressive and less boilerplatey language you'd have less lines. Plus all the poor architecture/spaghetti

One thing about your database, while splitting the boards, posts, threads, and files into separate tables is logical and common sense to the domain you realize to do metrics like showing the number of unique ips in a board, etc. you will have to be doing alot of joins + group by functions and possibly some window (over/partition by) as well, which will be pretty slow.

Maybe for reads on ip counts for example you could implement a materialized view or something, the problem is that ip counts typically display on the front page so if the query takes a ton of time with the grouping etc. it could make it vulnerable to DOS/refreshing.

Inserting a new thread - if there are constraints which say every post has to be associated with a thread, that means they will have to be inserted in both tables at the same time, like (pseudocode):

with insert1 as (<insert into threads table> RETURNING id AS threadid) INSERT INTO posts (id_thread <other fields>) VALUES ((select threadid from ins1),<other fields>)"

Pruning the catalog could be done with triggers on inserting to the threads table. Triggers could probably also auto lock threads with certain amount of responses.

your schema looks pretty good and logical, hope you don't mind if other people steal it :) not sure what the diff is between autosage and bumplock is thought?

just some suggestions again maybe youve already solved this