/tech/ - Technology

Soygeleno won btw.

>>31282
>>31283
so activitypub (+ some trick to obfuscate user identity) but with mentally retarded non-features: ai and hardcoded formatting

if it is federated and I want to setup an instance I have to also setup a local chatbot or pay a chatgpt commercial license? that's an absurd entry cost for what will be, at best, an extremely niche platform.

"link-rot protection" is always a meme because if I stop hosting something, it doesn't matter what the url was. if by link-rot you mean that content that is still hosted by the server becomes unreachable because of url changes, every single ib that stores files by their md5 hash (like 4chan) already has that feature. matter of fact, this is only a problem in enshittified corporate internet. do you also need to specify that your ib won't be vulnerable to script injection?

>>31284
>kitchen sink chat/bbs/wiki/booru
I think there is a scope, but it might be too big. Maybe the scope is rather a purpose, it is what it does. Anything which would improve my usage of my (and others') zettelkasten. [^1] (To put briefly something put much better by me in a different format today) I'd like to improve thinking.

>i try to minimize the set of core features while supporting as wide of a featureset as possible
Guess the challenge is in designing this… Am struggling to imagine what to get rid of!

>>31286
>hardcoded formatting
This is a very interesting critique to me. I don't think it would federate well if you had any options for formatting. You could certainly allow for customizable CSS and scripts though, just like a static site generator.

>if it is federated and I want to setup an instance I have to also setup a local chatbot or pay a chatgpt commercial license?
No, this was going to be an optional feature, disabled by default. :[^1] If there was some tool to help me finish books I've started taking notes on that would be even better, but this and the social impetus (of publishing) to do the right things, are the only things to mind that could encourage me to use a zettlekasten better.

>if I stop hosting something, it doesn't matter what the url was.
Haven't worked out the concept yet but it would be nice to have local archives of linked markdown pages (if this is recursive it could cause trouble). I believe foreign replies would be locally archived by default since they're sent to the inbox. And another idea was to send a reply event every time there's a markdown link for backlinks (but this would be impossible to do recursively).

>>31287
>Don't think it would federate well if you had any options for formatting.
This could be expanded to the Mastadon tag set which is reasonably large other than not including <div> or <table>…

>>31288
So if you make an Article instead of a Note you get unlimited length and HTML tags because you just link back to your instance in the feeds. Am reminded of my hatred for traditional WYSIWYG, and the following from six years ago is accessed in memory (pic rel): https://secretgeek.github.io/html_wysiwyg/html.html Bet could have an HTML editor that, like Obsidian LaTeX rendering, is rendered typically when the cursor is outside of them, and otherwise in this "brutalist" view. How hard could it be?

>>31291
Was playing around some more with this idea.
- Walk the tree
1. getting the tag name and attrs, setting :before and :after as needed to a generated class name.
2. setting tag decorator position: absolute and opacity: 0 by default
3. applying contenteditable=true to everything.
- When the cursor is inside or adjacent a tag display it. (only implemented currently on :hover)
- On insert create a new Node if that's necessary
- Some interface for editing existing tags???

This sort of thing gets a little hairy for more complicated websites.
Might be nice if the decorator was overlaid.
You could then have a toggle with perhaps a mode for all, or just the inside/adjacent.

Think the main trouble is that styling (mainly class) and boxing (div and span) are mixed in with the semantic tagging.
(The most trivial to remove is the contenteditable="true" which shouldn't have been there in the first place.)
Removing the attrs entirely by default is a massive improvement.
So yes, HTML is markup, you don't really need the translation layer, but it's also it's not really semantic or light.
On the other hand this sort of styling and boxing is only required for websites that are more complex than possible with markdown.
There could be an HTML to HTML static site generator in JavaScript for table of contents, MathJax, and similar.

>>31296
Before i tried wrangling HTML into an embeddable markup format, i would just bite the bullet and define my own markup dialect and write transpilers from a HTML subset, bbcode, etc.

Look, if you can't think of an elegant base framework for your features, i'll try doing the work for you:
An article contains a body (content), terminal tags (title, date, flag, uploaded files) and non-terminal tags (forum-style tags, thread number, thread number, tripcode). Non-terminal tags define relations between articles, allowing a client to submit posts to a thread on a given board or display all threads in bump order. Tripcodes act as a secure identifier and therefore allow user accounts or pages by assigning articles to an author or even submit a twitter-style thread under a disposable trip.

I think the above definition is concise enough to be a small subset of something like activitypub (the only "activities" would be submitting articles and retrieving articles), but general enough to implement everything you mentioned. User sessions, link walking and display would all be malleable, i feel like encoding all of this into a stripped-down format goes a long way though. Now it's your turn to build the next layer!

>>31298
To give you an idea about what this would look in practice, i will now define a few of your features on a very limited set of non-terminal tags: board, thread and tripcode:

The server is free to reject article submissions to boards or thread that have no previous representation or "don't exist" in whichever representation the server chose. The absence of a thread tag assigns the article a free thread id. Tripcodes act as user identifiers, therefore there can be a convention of showing the latest thread under the trip on /~users/ as a user page (allowing only one thread id per trip on this board and overwriting it on new submissions would make this more convenient). 'Recpients', a terminal field containing a list of tripcodes, would allow the server to discriminate who could retrieve articles, which would be necessary for implementing private messages (i would also dump these to a special purpose board).

>OP moderated "threads"
This one is tricky, but the most straightforward approach would be submitting to a moderation queue board and having the OP "resubmit" accepted articles.
>distinction between zettlekasten and bibliographic notes
Specify a terminal tag for each.

This scheme is fairly kludgy, but i think it gets the point across.

>>31298
The idea that's floating in my head (now) is to not embed HTML, but the application, as a set of optional widgets.
One widget displays backlinks as Article links, and replies as Notes, and allows for the creation of Notes (maybe three tags for these).
Another takes an inbox, outbox, or sharedInbox and creates a feed.
The base front-end for the application is a nearly full-screen modified contentEditable with a submit/edit button.
Authentication might be a little tricky… but should be possible…

I do think the distinction between relational data and formatting data is valuable.
Even though this is a smaller application, it still feels like it could be smaller.

>>31298
The parsing analogy is interesting. Think it's understood. Wonder if it generalizes?

>>31299
So this is about API design right?
You're saying there should be, in this case, a single endpoint.
With hidden fields in the form and the action attribute set.
The point is then to reduce the complexity of the transmitted data, endpoints, and render.
All while achieving a certain level of civilization.

This makes me want to get back to rewriting arsvia2, instead of idly contemplating redux.

>>31298
>>31299
Used this to delete a table, and simplified and renamed a few more. Similar effects on simplifying classes. Also consolidated the queries, so that there's really only two written now, but they service the tripcode, address (for admin), thread, catalog, and recent views. Think there might be three other pages so perhaps three more smaller queries to come. Think getting rid of the Mixins is the next big step for the rewrite.

Today only managed to get nested contentEdtable elements to work like they should.
The key idea is to zoom out the lens (all the way) for navigation and zoom in (all the way) for editing.
Focusing the body with addEditable is necessary for repeat keypresses.

<!DOCTYPE html>
<html>
<style>
*:focus { outline: none; }
</style>
<body>
<div id="outer">
some text
<div id="inner">
inner
</div>
more text
</div>
</body>
<script>
const body = document.body;
const outer = document.getElementById('outer');
const inner = document.getElementById('inner');

inner.addEventListener('mouseup', () => narrowToSelection(), true);
inner.addEventListener('touchend', () => narrowToSelection(), true);
outer.addEventListener('mouseup', () => narrowToSelection(), true);
outer.addEventListener('touchend', () => narrowToSelection(), true);

const isArrow = k => ['ArrowLeft','ArrowRight','ArrowUp','ArrowDown'].includes(k);

function addEditable (node) {
  node.classList.add("_arsvia_editable");
  node.contentEditable = true;
  node.focus()
}

function removeEditables () {
  Array.from(document.getElementsByClassName("_arsvia_editable"))
  .forEach(el => {
    el.classList.remove("_arsvia_editable");
    el.removeAttribute('contentEditable');
  });
}

function narrowToSelection () {
  removeEditables()

  const selection = window.getSelection();
  if (!selection.rangeCount) return;
  let narrow = selection.getRangeAt(0).startContainer;
  if (narrow.nodeType === Node.TEXT_NODE) narrow = narrow.parentElement;

  addEditable(narrow);
}

window.addEventListener('keydown', e => {
  if (!isArrow(e.key)) return;

  removeEditables();
  addEditable(body); 
});

window.addEventListener('keyup', e => {
  if (!isArrow(e.key)) return;
  narrowToSelection();
});
</script>
</html>

Other than this worked on the HTML editor some.
Had the machine replace the decorators with actual content that's hidden so that they can be edited.
Still trying to workout how to have it autoupdate the surrounding tag, and deal with insertions, and attributes.
Further not sure how to deal with metadata, scripts, and styling in an elegant way.

>>31313
The reason wrote this was to have an input event fire on the "inner" element.
A far better way to do this is to have the input event look for the commonAncestorContainer of the range.
This is now the entirety of what remains of the lens logic.
The machine found an different way to replace the actual decorator with contents.
That is to have the decorator outside the tree and only position it absolutely when it needs to appear.
There's a small invisible spacer that determines where they are to go in the tree.
Ideally even this spacer wouldn't be in the tree and it's pretty annoying that it's there.
The machine wasn't very helpful other than this, and now seem to barely remember why this helped.
It was something to do with the cursor not appearing in the correct position before if it was 1ch wide.
Got autoupdating of the surrounding tag, and attributes to work also, although attributes are no longer hidden.
A previous version also had the insertion working, but this is no longer the case.
Am starting to be a bit warn from the LLaMa shaving.
Much prefer just programming, and taking things a little slower, but can't seem to.

>>31325
>Ideally even this spacer wouldn't be in the tree and it's pretty annoying that it's there.
Thought of (and wrote) a way around this just positioning absolutely on lines from inner to outer if there's too many tags to display where they belong.
Also managed to get a primitive version of the HTML insertion working (with undo redo), think the behavior that's desired is for '<' to insert '<><\>' with the cursor in the tag.
Am currently working on the display of self-closing tags and maybe head information.
Decided the CSS editor should be a simple button at the end of the tag which opens a popup, ideally rather like the dev tools editor.
That's about it for today.

>>31329
>Thought of (and wrote) a way around this just positioning absolutely on lines from inner to outer if there's too many tags to display where they belong.
This works pretty great even for more complicated MathML expressions, the (now less literally) nested lenses only show what's required.

>Am currently working on the display of self-closing tags[.]
This now works also.

Inserting tags still doesn't work like would like it to, and there's no way to display the head section.
A cheap solution would be to just always show the head section, and similar, or have them toggleable.
Inserting the tags could work nearly as it does now if the tag didn't eat the remainder of the text block on input.

>>31330
Amusingly this interface as implemented is incompatible with editing of <head>, <script>, and <style>.
The best solution have thought of is to have a button and popup for editing tags literal content, including <html>.

Considering scrapping this editor, and trying to simplify…

Might use the following tools:
- plain .shtml documents.
- <template></template> tags, and (authorside) javscript.
- <meta name="keywords" content="psychology, buddhism" />

- (authorside) scripts to generate navigation, backlinks, tag pages, feeds, etc.
- a client side script to fetch fediverse comments, popups, etc.
- an (SSI) HTML LSP server for completion of links, following backlinks, meta keywords, etc.
- an AI LSP for the "integrate knowledge" button.

So basically a HTML blog and some tools for writing it…

To publicly host arbitrary articles there'd need to be some small number of additional scripts:
- There'd need to be a recursion depth check (perhaps no recursion) to host arbitrary SSI HTML or force authorside rendering.
- It could have a git based interface so that it just clones –depth=1 repos when prompted (statichost.eu).
- We really just want to publish to existing fediverse servers rather than host one so users posting uses their existing accounts. (fed.brid.gy)
- No real front-end is required beyond registering, and setting the git repo. (statichost.eu)

Other than using (SSI) HTML there are existing solutions for all this.
If I wanted to user Markdown, even the HTML tooling would go.
Overall pretty disappointing.

>>31346
How many people do you expect people to buy into your new document format? I would instead either try to start from an existing document pipeline like docbook/asciidoc or make it possible to embed common document formats and write scrapers for metadata and such (maybe even hijack internal links to hook back into your platform).
Flood detected; Post discarded.

>>31348
It's all off-the-shelf [^1][^2][^3], but that's rather irrelevant since no one uses the stuff.
Guess the only tricks here are the LSPs for writing HTML, if that's even helpful.
It could be hosted with any VPS, Nginx, and Hatsu [^4] or fed.brid.gy.

[^1]: https://en.wikipedia.org/wiki/Server_Side_Includes
[^2]: https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/template
[^3]: meta keywords also exist unofficially but all the articles on it are SEO spam.
[^4]: https://hatsu.cli.rs/

Made this stupid editor work more or less completely.
Coming in at <400 lines, it's certainly a featherweight.
I didn't add the CSS button so, it's strictly for HTML.
The key was to punch in zero-width spaces all over the place.
And of course clean them up when we're done with them.
Well that and a new prompting methodology… >>31361

>>31362
This is so close, but can't quite get it perfect.
In such times it's often best to take a break.

Have been thinking about my imageboard's actual board.
The idea is to just start with one with broad scope.
This way the small user base is as undivided as possible.
It's requires fewer tabs for the users so less burden for them.
It also provides a theme that the imageboard can build on.

This initial board is to be called /b/ - timely.
The board is to discuss all things future:
technology, programming, AI, sci-fi anime, (relevant) political happenings, science, etc.
The forbidden content is roughly as follows:

1. trivial
- repetitive
- irrelevant
- vacuous
2. unfalsifiable: insufficient provisioning of contextual "why".
3. misrepresenting: representing another in a way other than how they perform.

The main weaknesses can think of are pseudoscience, futurism, and false flags.
Can't really think of a way to remove these things without self-moderation of some sort…
Which is out of the question.
Perhaps filtering these is some part of the discussion…

>>31362
>>31371
>This is so close, but can't quite get it perfect.
In such times it's often best to take a break
Editors are brutal man

Eventually to fix those uyghling issues you generally have to start thinking about a separate internal representation of the text

>>31372
In some ways this is barely even an editor.
More of an HTML to HTML transformation to make the existing editor work.
And to display some metadata about the HTML.
There's been just a little too little planning for it to work out correctly.
And because of a lot of edge cases I didn't know about.
Mostly that there are tags which can't be inserted into or which can only take certain elements.
And that you have to insert something into a tag to navigate inside of it as a contenteditable.
But also that you can't edit the tag name of a tag without deleting and recreating it.
And there were probably others too.

>>31373
May have just perfected this little editor, but the implementation is terrible.
I've attached it for enjoyment by the general public.
These laughs are on the house.

I'm not reading this thread rn, but what about a TUI imageboard? Modern terminal emulators and TUI libraries are capable of even displaying images. That way you could ssh into the board? Idk it's not a very fleshed out idea and i have little experience.

>>31377
There's one or two of these, think of them sometimes.
There's also the tildeverse. which is sort of similar.

Am thinking about making arsvia2 an textboard with drop caps.
This is due to the pricing of image content filtering at $1/20 images.

Had a productive programming session today.
Implemented nearly the entire remainder of the backend for arsvia2.
Probably just a day or two left of work on it.
Realized NGiNX can be used for caching and flood detection.
There were some other pleasant surprise.
Still need to write the two or three websocket endpoints.
The two permission checks in the application, and TRIGGERs.
Oh and also the /board/index endpoint and query, which was absent from 1.0.
TRIGGERs include bumping, bumplocking, and applying reports/bans.

Was thinking some of the front-end too, and thought might go for a modern implementation.
Might actually implement the recursive post preview, and post from index.

>>31393
Was a productive day, implemented trip_code diaries, liveboard websockets, index, and actually started running the backend.
As part of scrubbing the API, am considering folding four GET endpoint into one POST endpoint since they all return post lists.
Then might as well make them all POST since having one GET would be weird, but this would be a little strange too!
So far have done manual testing of the index, thead, catalog, and post endpoints. They all seem to be working.
Tomorrow I'll work on testing address, trip_code, reports, report, and ban endpoints.
Also I haven't yet set back up the admin panel. Not sure how

>>31403
Manually tested address, trip_code, and reports.
Am using the trip_code as an Authentication Bearer in the API (there are no query parameters).
Made the Bearer optional for /api/report and /api/ban to apply these.
Am going to change these endpoints to /api/user_action and /api/post_action.
Probably need to add an endpoint for listening for user_actions and rename report.
The request will be structured as a ADT in Pydantic, so only necessary data is required.
This way the admin panel is just (hidden) JSON.stringify <form></form>s on plain pages.

>>31413
Have run into some trouble with trying to setup doomscrollability in the API.
And also constraints on how the liveboard features effect the API.
And further had to switch around the address endpoint to be a related posts endpoint.
This is to make it clickable in the front-end without requiring an identifier for the poster.

- /api/{board}/reports?t={time}&o={offset}&l={limit}
- /api/{board}/catalog?t={time}&o={offset}&l={limit}
- /api/{board}/index?t={time}&o={offset}&l={limit}
- /api/{board}/res/{thread}?t={time}&o={offset}&l={limit}
- /api/{board}/rel/{post}?t={time}&o={offset}&l={limit}
- /api/{board}/trip/{trip_code}?t={time}&o={offset}&l={limit}

This means in all the queries for the GETs have to recompute the bump at {time}.
This shouldn't be too difficult, but it's not like two days ago.

>>31427
This wasn't so bad. Just some relatively small SQL changes.
Renamed the /api/ban endpoint to /api/moderation, and still need to write the ADT for it.
The post endpoint handler is getting a little large.
It might be necessary complexity though.
It's nine queries (including inserts (otherwise used elsewhere)), and a hundred lines.
The bulk of this is fetching data to, and updating the websockets for five pages.

>>31428
Once started to try to keep deletions deleted this turned out to be a about as difficult as expected.
The tricky thing is that there can be a deletion less than {offset} after {time} and then you get duplicates.
So you've got to add all the extra deletions to the offset to get the proper offset.
Got that ADT written for the moderation endpoint, but haven't written the handler.
Think after get this project deployed I'm going to start looking for jobs again (maybe IT, non-support if that exists).

>>31431
My application is a bit of a mess now.
NGiNX can in addition to caching and rate limiting do per URL file size, and mime checks.
Am working on separating out a /api/file application/json POST internal endpoint.
The actual endpoint presented by NGiNX will likely be /api/{board}/file.
The trickiest bit seems to be the garbage collection of failed uploads.
Still haven't finished the /api/moderation endpoint or the deletion offset correction.
Think when all of this is set, then it will be time to move on to the front-end.

>>31433
Even the renaming to the hash, and thumbnail generation can occur in NGiNX with a little lua.
This would have required me to change the API slightly and is ugly in itself so I've ruled it out.
The only reason nearly was able to justify it is that the endpoint is already going to be different.
Still need to figure out the whole garbage collection thing for files not connected to posts.
Just set up incremental copy and hashing for files which should drop memory usage some.

Am going to try to start saging this thread unless it slides off page one.

Did one round of fixes on the endpoints and queries just to break them again.
But the timetraveling with deletes problem mentioned in >>31431 was solved.
Pulled out all the mod options into separate junction tables by type.
Think now have a sketch of how moderation options should work in general.
Removed all reference to references in the backend which simplified things some.
Switched from Bearers to HTTP Basic for all the trip_code protected endpoints.
Switched from post IDs to Base64 encoded Snowflakes (that's twelve characters).
Also decided to have server side hidden posts, like with the triangle menu.
These can also apply to IPs to allow for something like shadow banning.
Am also considering a fairly massive rewrite of the API for cachability.

Was thinking quite a lot about moderation issues.
Excluding hCaptcha for captchas there are apparently human farms from 0.005 USD a captcha.
You can apparently buy an IP for a month from a botnet for 0.05 USD.
Using Spamhaus XBL you can block somthing like 80% of botnets so in a perfect world 0.25 USD an IP.

Overall have found the space a little disappointing.
Basically 0.255 USD a spam post unless JavaScript is mandatory.
And really even seems difficult to perform moderation actions at all with Tor and VPNs permitted.

Could just use CloudFlare, hCaptcha, and block VPNs/Tor.
This would cut the Gordian knot, but it hurts me a little too.

>>31482
>Am also considering a fairly massive rewrite of the API for cachability.
>JavaScript is mandatory.
This is turning out to be a little more significant than first envisioned.
Could make every post into (cachable) JSON cached by a CDN.
Further could use a serverless frontend hosting a Preact SPA.
Because a certain provider has more bandwidth than they know what to do with this is free.
Lastly could make use of GPU-accelerated image transformation hosts.
The server handles the validation, parsing, and GET/socket sending snowflake vectors, or from the CDN.
This hybrid-serverless approach seems to play the economy right.

>>31483
>This is turning out to be a little more significant than first envisioned.
Got the endpoints rewritten and am most the way through with the models.
As usual its the moderation related functionality that are giving the most trouble.
Further need to rework the related posts endpoint to be able to handle range bans.

>>31487
>Got the endpoints rewritten and am most the way through with the models.
>As usual its the moderation related functionality that are giving the most trouble.
>Further need to rework the related posts endpoint to be able to handle range bans.
Finished off the endpoints and the models, had to add back References.

An advantage of ">>thread/post" syntax is that you can shard on board without querying across shards.
This is because the parser no longer needs to query for the thread of the post to link.
However with snowflakes this would be 22 or 24 characters long, which is unreasonable.
Further removing this step from the parser also removes the validation of cross-thread links.

We could mangle the endpoint such that /{board}/res/{post} returns the relevant /{board}/res/{thread}.
The front-end would then be responsible for translating to the canonical /{board}/res/{thread}#{post}.
Since it's already an SPA this wouldn't be the biggest deal, but is ugly, and removes cross-thread link validation.
The validation isn't as big a deal as the ugliness to me.

Trips are another problem in need of a solution…

>>31495
>We could mangle the endpoint such that /{board}/res/{post} returns the relevant /{board}/res/{thread}.
Made the backend require an SPA or complex DOM operations in the front-end by making links unusable without JavaScript.
The reason was to make sharding by board trivial, avoiding the parser having to query for post threads to form URLs.
Corrected my error and decided returning a correct output for the user was more valuable than more efficient sharding.
At present whatever handles the front-end just has to be capable of fetching/manipulating JSON, and concatenating strings.
Rendering the HTML contained in the JSON in native apps isn't much more complicated given existing libraries.

Regarding the HTTP header waste caused by having each post be requested separately rather than a bulk endpoint.
For an English language textboard waste could be as high as 10%, but for an imageboard more like 1%.
But this is waste that doesn't touch the server since all the posts are sent from the CDN to the client.
I'm worried about cache poisoning when hosted without a CDN or without cloudflare;

>>31497
It feels like the API is starting to solidify.
Only successful API changes of the last several were:
- Replacing the offset with a cursor.
- Allowing for address ranges in the related posts.
Am working on the queries presently.
Completed the thread, and catalog queries.
Even optimized them a little, though don't really know how.
Was pretty weak today, and didn't get much out of bed.
So the rest of the queries are going to probably wait.

>>31503
>So the rest of the queries are going to probably wait.
Managed to get a few more similar queries done:
So now there are trip, related, and report queries.
This included setting up IP range queries for the related page.
These work without exposing the IP to the moderator.
Especially helpful if there are ever user created boards.
Also compressed down the JSON for the post metadata.
The vast majority of the data here is null.
So we just remove the empty elements.

>>31506
Am interested in transforming this into a federated platform with "little boxes".
There are mandatory trips where the trip password is a edDSA private key and the trip a public key.
These are generated client side, and the private key never hits the wire let alone the server.
Usernames are equal to the public key, unless the user changes them.
The client signs the HTTP packet client side that will be sent to the outbox of the post it responds to.
There are group actors of different kinds which make up the boards.
We accept all follows and drop all DMs to keep things simple and safe.

>>31508
>Am interested in transforming this into a federated platform with "little boxes".
There are three problems with this:
1. Fediverse UI assumes one repliee per post.
2. There are caching and scaling difficulties with ActivityPub.
3. ActivityStreams are complicated.

Fortunately 1. is a small percentage of posts.
Probably less than 5% are of a form that can't be easily broken up or changed for an @mention.
In the absolute worst case you should still be able to link to posts using the URL.
Still not sure about what the UI for this should be. [^1]

Seems the standard issue scaling solution is making everything maxage=300 JSON plus a CDN.
The Inbox is a POST endpoint, and so can't be cached, but there is sharedInbox which makes it a little easier.
Being focused on Group with anonymous users should prevent the feed, and search related performance drains.

Sounds like a ton of work if not another complete rewrite…

Unrelated but ended up combining WikiMedia, Markdown, and old Reddit markup to make a hybrid that seems to work well:
Wikimedia: labeled external URLs, and italics
Markdown: quotes, bold, underline (sort of anyway), strike, code
old-Reddit/StackOverflow: spoilers

wikimedia_external_link = (string("[") >> url << string(" ")) + \
    formats_without_urls.until(string("]")) << string("]").map(
        lambda url, label: LabeledURL.model_validate({ "url": url, "content": label })
    )

pre = (string("```") >> pretext << string("```")).map(
    lambda c: (make_code(c)).model_validate({ "content": c })
)

quote = (regex(r"(^|\n|\r)>") >> formats.until(regex("(\n|\r|$)"))).map(Quote)

spoiler = balanced(">!", "!<", Spoiler)
strike = balanced("~~", "~~", Strike)
italic = balanced("''", "''", Italic)
bold = balanced("**", "**", Bold)
underline = balanced("__", "__", Underline)

This is bound to be confusing, so I'm going to have a popup on first login show the format and rules.

:[^1] Might be misremembering this but seem to recall a web forum with parent child color correspondence.
So each post is assigned two colors, one as a parent and one as a child and if the child-parent color and parent-child color match they are related.
This keeps the replies flat (there's only ever two color blocks) while still having a visual graph.
You could also have multiple views for chronological versus replyTo chronological sorts.
We keep post preview on hover also, which is a real quality of life improvement for non-local replies.
Then again maybe this is strictly worse than just using the SnowflakeID…

>>31540
Less gibberish, important bits:
- Keep private keys off the wire/server.
- Cache immutable the outbox.
- Use sharedInbox.
- Use a CDN for caching.

Also with https://github.com/joewlos/activitypubdantic instead of "little boxes".
And further drop any strange pagination requests to the outbox.

>>31541
Thought of a couple more ways to improve performance.
- Use workers for fanout of POST requests in server-to-server.
- Use NoSQL for storage of ActivityPub.

Am using Beanie in this 2.2, and the question is, "is this an added translation layer".
There is imperfect translation between Beanie and PyMongo Async queries.
This goes against a design principle that was working so excellently in 2.1.
And this is a little disappointing.

The plus-side is managed to make it economical to scale since it's 0.30 USD / 1m requests.

>>31541
>Cache immutable the outbox.
I wrote signature verification for the inbox and three implementation for the Person outbox.
Think I'd like it to be illegal to use an irregular cursor/page so that the cache rarely misses.
But would also like to avoid using the "skip" parameter with O(n) search of the documents.
No matter how it's implemented it seems to require a document mapping cursors to pages.

>>31548
Settled on just using a cursor and "trusting" that servers won't use irregular cursors.
Or else that it may be possible to remove services which query the origin excessively.

Further this is a singly linked implementation because Delete and Update are included.
Delete and Update requires that the pages be traversed in full to render next elements.

The first page is always the total_items modulo the config.OUTBOX_PAGE_SIZE.
This allows every subsequent page to be cached immutable so long as the linked cursors are used.

We also drop the "partOf" parameter to avoid making the full (mutable) collection.

It's all above board with the spec too.
Only downside is the mentioned "trust" required of servers.

Wrote the Person following and followers endpoints.
Ended up not materializing the document to keep track of this.
So similar to the outbox this is just a query on the ActivityModel class.
The only real advantage to this is in bookkeeping.
It's slow because it's not really possible to cache these pages.
It should be less than 50ms (maybe less than 10ms on heavy hardware), for a page, which is probably too slow.
There is also a precondition that there be one follow not undone for any thousand.
This is to make it computationally feasible.

>>31567
It's a bit of a fail to write a federated server with mongodb in anything but typescript.
Guess there's going to be a 2.3 using fedify, mongodb, and typescript.
Think need to separate out the Activity logs from the materialized views.
This is to make the follwers, following, and like sufficiently performant.
My impression is that the client to server protocol would make things like bump ordering difficult.
So there probably needs to be a third layer to the API for a cache efficient client.

>>31583
For future reference, the private key idea is to use fordwardActivity() [^1].
And simply sign on the client side for all the relevant servers sent via a separate endpoint.
It's apparently trivial to wrap the existing fedify classes as monogodb documents with indices.
For the POST fanout use fedify/x/cfworkers [^2] including POSTing to the origin…

:[^1] https://fedify.dev/manual/inbox#forwarding-activities-to-another-server
:[^2] https://github.com/fedify-dev/fedify/pull/242