#swad

Felix Palmen :freebsd: :c64:This redesign of <a href="https://mastodon.bsd.cafe/tags/poser" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poser</a> (for <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a>) to offer a "multi-reactor" (with multiple <a href="https://mastodon.bsd.cafe/tags/threads" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#threads</a> running each their own event loop) starts to give me severe headaches.There is *still* a very rare data <a href="https://mastodon.bsd.cafe/tags/race" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#race</a> in the <a href="https://mastodon.bsd.cafe/tags/lockfree" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#lockfree</a> <a href="https://mastodon.bsd.cafe/tags/queue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#queue</a>. I *think* I can spot it in the pseudo code from the paper I used[1], see screenshot. Have a look at lines E7 and E8. Suppose the thread executing this is suspended after E7 for a "very long time". Now, some dequeue operation from some other thread will eventually dequeue whatever "Q->Tail" was pointing to, and then free it after consumption. Our poor thread resumes, checks the pointer already read in E6 for NULL successfully, and then tries a CAS on tail->next in E9, which is unfortunately inside an object that doesn't exist any more .... If the CAS succeeds because at this memory location happens to be "zero" bytes, we corrupted some random other object that might now reside there. 🤯 Please tell me whether I have an error in my thinking here. Can it be ....? 🤔 Meanwhile, after fixing and improving lots of things, I checked the alternative implementation using <a href="https://mastodon.bsd.cafe/tags/mutexes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#mutexes</a> again, and surprise: Although it's still a bit slower, the difference is now very very small. And it has the clear advantage that it never crashes. 🙈 I'm seriously considering to drop all the lock-free <a href="https://mastodon.bsd.cafe/tags/atomics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#atomics</a> stuff again and just go with mutexes.[1] <a href="https://dl.acm.org/doi/10.1145/248052.248106" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://dl.acm.org/doi/10.1145/248052.248106</a>

Felix Palmen :freebsd: :c64:I just stress-tested the current dev state of <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> on <a href="https://mastodon.bsd.cafe/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a>. The first attempt failed miserably, got a lot of errors accepting a connection. Well, this lead to another little improvement, I added another static method to my logging interface that mimics <a href="https://mastodon.bsd.cafe/tags/perror" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#perror</a>: Also print the description of the system errno. With that in place, I could see the issue was "too many open files". Checking <a href="https://mastodon.bsd.cafe/tags/ulimit" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ulimit</a> -n gave me 1024. Seriously? 🤯 On my <a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a> machine, as a regular user, it's 226755. Ok, bumped that up to 8192 and then the stress test ran through without issues.On a side note, this also made creating new timers (using <a href="https://mastodon.bsd.cafe/tags/timerfd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#timerfd</a> on Linux) fail, which ultimately made swad crash. I have to redesign my timer interface so that creating a timer may explicitly fail and I can react on that, aborting whatever would need that timer.Anyways, the same test gave somewhat acceptable results: throughput of roughly 3000 req/s, response times around 500ms. Not great, but okayish, and not directly comparable because this test ran in a <a href="https://mastodon.bsd.cafe/tags/bhyve" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bhyve</a> vm and the requests had to pass the virtual networking.One major issue is still the <a href="https://mastodon.bsd.cafe/tags/RAM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#RAM</a> consumption. The test left swad with a resident set of > 540 MiB. I have no idea what to do about that. 😞 The code makes heavy use of "allocated objects" (every connection object with metadata and buffers, every event handler registered, every timer, and so on), so, uses the <a href="https://mastodon.bsd.cafe/tags/heap" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#heap</a> a lot, but according to <a href="https://mastodon.bsd.cafe/tags/valgrind" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#valgrind</a>, correctly frees everything. Still the resident set just keeps growing. I guess it's the classic <a href="https://mastodon.bsd.cafe/tags/fragmentation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#fragmentation</a> issue...

Felix Palmen :freebsd: :c64:<a href="https://mendeddrum.org/@fanf" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@fanf</a> Sure that does make sense. I'll try to verify jmeter indeed doesn't reuse connections (I already have debug logging in place that should tell me).If that's really the reason, I guess the sane thing to do is to add a hint to the docs to just disable TLS for very busy sites. The intended usecase for <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> is operation behind <a href="https://mastodon.bsd.cafe/tags/nginx" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#nginx</a> to serve its "auth_request". I don't intend to implement HTTP/2 or beyond, but it would be pretty pointless here anyways, nginx defaults to HTTP/1.0 for proxy requests and can be configured to use HTTP/1.1 instead, but *still* doesn't reuse connections by default, and my experiments so far to enable it weren't successful, maybe I didn't fully understand it yet. Using TLS behind nginx would make sense from a "defense in depth" point of view, but it's probably impractical once your load exceeds a certain threshold.For background how I arrived there, I observed stupid <a href="https://mastodon.bsd.cafe/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://mastodon.bsd.cafe/tags/scraper" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#scraper</a> <a href="https://mastodon.bsd.cafe/tags/bots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bots</a> clog my DSL connection by downloading gigabytes of build logs produced by my <a href="https://mastodon.bsd.cafe/tags/poudriere" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poudriere</a>. They're not secret in any way and having a simple way to share them is great for community bug hunting, but this had to stop. I had a simple C library doing a fully portable reactor event loop on top of select (so, not really scalable), and some very limited HTTP/1.1 server code from experiments with TOR hidden services ... so I put that together to add some web-form + cookies auth to my private nginx to lock out the bots. Later, I added a "guest login" doing the same "proof of work" stuff known from <a href="https://mastodon.bsd.cafe/tags/anubis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#anubis</a>, and then I suddenly had the idea in mind to make my little service (that already solved the problem perfectly for myself) suitable for large-scale installations. So, added kqueue, epoll etc support, added a "multi-reactor with acceptor-connector" design, etc .... and now I'm a bit frustrated enabling TLS spoils all the performance 🙈

Felix Palmen :freebsd: :c64:More interesting progress trying to make <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> suitable for very busy sites!I realized that <a href="https://mastodon.bsd.cafe/tags/TLS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TLS</a> (both with <a href="https://mastodon.bsd.cafe/tags/OpenSSL" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSSL</a> and <a href="https://mastodon.bsd.cafe/tags/LibreSSL" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LibreSSL</a>) is a *major* bottleneck. With TLS enabled, I couldn't cross 3000 requests per second, with somewhat acceptable response times (most below 500ms). Disabling TLS, I could really see the impact of a <a href="https://mastodon.bsd.cafe/tags/lockfree" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#lockfree</a> queue as opposed to one protected by a <a href="https://mastodon.bsd.cafe/tags/mutex" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#mutex</a>. With the mutex, up to around 8000 req/s could be reached on the same hardware. And with a lockfree design, that quickly went beyond 10k req/s, but crashed. 😆So I read some scientific papers 🙈 ... and redesigned a lot (*). And now it finally seems to work. My latest test reached a throughput of almost 25k req/s, with response times below 10ms for most requests! I really didn't expect to see *this* happen. 🤩 Maybe it could do even more, didn't try yet.Open issue: Can I do something about TLS? There *must* be some way to make it perform at least a *bit* better...(*) edit: Here's the design I finally used, with a much simplified "dequeue" because the queues in question are guaranteed to have only a single consumer: <a href="https://dl.acm.org/doi/10.1145/248052.248106" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://dl.acm.org/doi/10.1145/248052.248106</a>

Felix Palmen :freebsd: :c64:Getting somewhat closer to releasing a new version of <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a>. I now improved the functionality to execute something on a different worker thread: Use an in-memory queue, providing a <a href="https://mastodon.bsd.cafe/tags/lockfree" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#lockfree</a> version. This gives me a consistent reliable throughput of 3000 requests/s (with outliers up to 4500 r/s) at an average response time of 350 - 400 ms (with TLS enabled). For waking up worker threads, I implemented different backends as well: kqueue, eventfd and event-ports, the fallback is still a self-pipe.So, <a href="https://mastodon.bsd.cafe/tags/portability" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#portability</a> here really means implement lots of different flavors of the same thing.Looking at these startup logs, you can see that <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a> (<a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a> and other BSDs) is really a "jack of all trades", being used for "everything" if available (and that's pretty awesome, it means one single <a href="https://mastodon.bsd.cafe/tags/syscall" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#syscall</a> per event loop iteration in the generic case). <a href="https://mastodon.bsd.cafe/tags/illumos" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#illumos</a>' (<a href="https://mastodon.bsd.cafe/tags/Solaris" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Solaris</a>) <a href="https://mastodon.bsd.cafe/tags/eventports" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#eventports</a> come somewhat close (but need a lot more syscalls as there's no "batch registering" and certain event types need to be re-registered every time they fired), they just can't do signals, but illumos offers Linux-compatible signalfd. Looking at <a href="https://mastodon.bsd.cafe/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a>, there's a "special case fd" for everything. 🙈 Plus <a href="https://mastodon.bsd.cafe/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a> also needs one syscall for each event to be registered. The "generic <a href="https://mastodon.bsd.cafe/tags/POSIX" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#POSIX</a>" case without any of these interfaces is just added for completeness 😆

Felix Palmen :freebsd: :c64:Finally getting somewhere working on the next evolution step for <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a>. I have a first version that (normally 🙈) doesn't crash quickly (so, no release yet, but it's available on the master branch).The good news: It's indeed an improvement to have *multiple* parallel <a href="https://mastodon.bsd.cafe/tags/reactor" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#reactor</a> (event-loop) threads. It now handles 3000 requests per second on the same hardware, with overall good response times and without any errors. I uploaded the results of the stress test here:<a href="https://zirias.github.io/swad/stress/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://zirias.github.io/swad/stress/</a>The bad news ... well, there are multiple.1. It got even more memory hungry. The new stress test still simulates 1000 distinct clients (trying to do more fails on my machine as <a href="https://mastodon.bsd.cafe/tags/jmeter" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#jmeter</a> can't create new threads any more...), but with delays reduced to 1/3 and doing 100 iterations each. This now leaves it with a resident set of almost 270 MiB ... tuning <a href="https://mastodon.bsd.cafe/tags/jemalloc" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#jemalloc</a> on <a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a> to return memory more promptly reduces this to 187 MiB (which is still a lot) and reduces performance a bit (some requests run into 429, overall response times are worse). I have no idea yet where to start trying to improve *this*. 2. It requires tuning to manage that load without errors, mainly using more threads for the thread pool, although *these* threads stay almost idle ... which probably means I have to find ways to make putting work on and off these threads more efficient. At least I have some ideas.3. I've seen a crash which only happened once so far, no idea as of now how to reproduce. *sigh*. Massively parallel code in C really is a PITA.Seems the more I improve here, the more I find that *should* also be improved. 🤪 <a href="https://mastodon.bsd.cafe/tags/C" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#C</a> <a href="https://mastodon.bsd.cafe/tags/coding" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#coding</a> <a href="https://mastodon.bsd.cafe/tags/performance" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#performance</a>

Felix Palmen :freebsd: :c64:Mid way implementing "multi-threaded <a href="https://mastodon.bsd.cafe/tags/reactor" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#reactor</a>" (multiple event loops in parallel) in <a href="https://mastodon.bsd.cafe/tags/poser" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poser</a> for usage in <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> and it's going like expected. Debugging heavily multi-threaded stuff is really a PITA. 🤯

Felix Palmen :freebsd: :c64:Just released: <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> 0.11 -- the session-less swad is done!Swad is the "Simple Web Authentication Daemon", it adds cookie/form <a href="https://mastodon.bsd.cafe/tags/authentication" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#authentication</a> to your reverse <a href="https://mastodon.bsd.cafe/tags/proxy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#proxy</a>, designed to work with <a href="https://mastodon.bsd.cafe/tags/nginx" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#nginx</a>' "auth_request". Several modules for checking credentials are included, one of which requires solving a crypto challenge like <a href="https://mastodon.bsd.cafe/tags/Anubis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Anubis</a> does, to allow "bot-safe" guest logins. Swad is written in pure <a href="https://mastodon.bsd.cafe/tags/C" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#C</a>, compiles to a small (200-300kiB) binary, has minimal dependencies (zlib, OpenSSL/LibreSSL and optionally libpam) and *should* work on many <a href="https://mastodon.bsd.cafe/tags/POSIX" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#POSIX</a>-alike systems (<a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a> tested a lot, <a href="https://mastodon.bsd.cafe/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a> and <a href="https://mastodon.bsd.cafe/tags/illumos" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#illumos</a> also tested)This release is the first one not to require a server-side session (which consumes a significant amount of RAM on really busy sites), instead signed Json Web Tokens are now implemented. For now, they are signed using HMAC-SHA256 with a random key generated at startup. A future direction could be support for asymmetric keys (RSA, ED25519), which could open up new possibilities like having your reverse proxy pass the signed token to a backend application, which could then verify it, but still not forge it.Read more, grab the latest .tar.xz, build and install it ... here: 😎 <a href="https://github.com/Zirias/swad" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/swad</a>

Felix Palmen :freebsd: :c64:Just released: <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> 0.10<a href="https://github.com/Zirias/swad/releases/tag/v0.10" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/swad/releases/tag/v0.10</a>Swad is the "Simple Web Authentication Daemon". If you're looking for a way to add <a href="https://mastodon.bsd.cafe/tags/authentication" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#authentication</a> (and/or proof-of-work access as known from <a href="https://mastodon.bsd.cafe/tags/anubis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#anubis</a>) to your <a href="https://mastodon.bsd.cafe/tags/nginx" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#nginx</a> reverse proxy -- without adding yet another reverse proxy -- swad could be for you! It's written in pure <a href="https://mastodon.bsd.cafe/tags/C" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#C</a>, has few external dependencies (just zlib, and optionally OpenSSL/Libressl and/or libpam) and compiles to a pretty small binary. It's designed for usage with nginx' 'auth_request'.Swad is tested on <a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a>, some basic functionality tests were also done on <a href="https://mastodon.bsd.cafe/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a> and <a href="https://mastodon.bsd.cafe/tags/illumos" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#illumos</a> (descendant from <a href="https://mastodon.bsd.cafe/tags/solaris" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#solaris</a>). It *should* build and work on most <a href="https://mastodon.bsd.cafe/tags/POSIX" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#POSIX</a>-alike systems.This release mainly brings performance improvements and a few bugfixes. It's now stress-tested with Apache jmeter, verifying it can deal with at least 1000 requests per second on my personal (somewhat limited) FreeBSD host machine.

Felix Palmen :freebsd: :c64:Got somewhere:<a href="https://github.com/Zirias/swad/commit/1bbd1e90ff0623d972e8b71c881f590112a9668b" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/swad/commit/1bbd1e90ff0623d972e8b71c881f590112a9668b</a>Now, no bot ever causes <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> to create a server-side session, at least from what I can observe in my logs -- these bots don't attempt any login!I also disabled usage of CSRF tokens for the login form, which I forgot to mention in the commit message. They strictly require a session and are pointless on a login form anyways.

Felix Palmen :freebsd: :c64:I now decided I'll at least aim for some middle grounds: Rework <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> so it only needs a (server-side) <a href="https://mastodon.bsd.cafe/tags/session" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#session</a> once a user is <a href="https://mastodon.bsd.cafe/tags/authenticated" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#authenticated</a>!This does have some implications, e.g. passing a redirect argument to the authentication endpoint won't work any more. But experimentation shows a workaround would be to use an "internal redirect" to the login endpoint in <a href="https://mastodon.bsd.cafe/tags/nginx" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#nginx</a>.We'll see where I end up. Having sessions only for authenticated users should reduce the need for server-side RAM significantly, so I hope 😉

Felix Palmen :freebsd: :c64:Good morning! ☕ Now that I can't find any other bugs in <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> any more, I'm thinking again about how I could improve it.Would anyone consider deploying it on a busy site right now? Either as a replacement for <a href="https://mastodon.bsd.cafe/tags/Anubis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Anubis</a> (proof-of-work against bots), or for simple non-federated <a href="https://mastodon.bsd.cafe/tags/authentication" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#authentication</a>, or maybe even both?I'm currently not sure how well it would scale. The reason is the design with server-side sessions, which is simple and very light-weight "on the wire", but needs server-side RAM for each and every client. It's hard to guess how this would turn out on very busy sites.So, I'm thinking about moving to a stateless design. The obvious technical choice for that would be to issue a signed <a href="https://mastodon.bsd.cafe/tags/JWT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#JWT</a> (Json Web Token), just like Anubis does it as well. This would have a few consequences though:* OpenSSL/LibreSSL would be a hard build dependency. Right now, it's only needed if the proof-of-work checker and/or TLS support is enabled. * You'd need an X509 certificate in any case to operate swad, even without TLS, just for signing the JWTs. * My current CSRF-protection would stop working (it's based on random tokens stored in the session). Probably not THAT bad, the login itself doesn't need it at all, and once logged in, the only action swad supports is logout, which then COULD be spoofed, but that's more an annoyance than a security threat... 🤔 * I would *still* need some server-side RAM for each and every client to implement the rate-limits for failed logins. At least, that's not as much RAM as currently.Any thoughts? Should I work on going (almost) "stateless"?

Felix Palmen :freebsd: :c64:Found and fixed two more bugs affecting only <a href="https://mastodon.bsd.cafe/tags/TLS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TLS</a> with <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a>, so here's yet another "bugfix release":<a href="https://github.com/Zirias/swad/releases/tag/v0.9" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/swad/releases/tag/v0.9</a>One of these bugs was always there and I never noticed (just ignoring intermediate certificates) because many clients cope well with this, but not all.The other bug is yet another regression from earlier performance improvements. 😞So, lots of releases these last days. I'll have to remember to do very thorough regression testing whenever "optimizing" things in existing code 🙈 In a nutshell: 0.8 was finally fine again without TLS, but if you need TLS, better use this new 0.9.

Felix Palmen :freebsd: :c64:Unfortunately, I had to do a bugfix release: <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> 0.8Although I didn't observe any obvious misbehavior on my own installation for several days, I discovered two very relevant bugs just after release of 0.7 🤦‍♂️ -- one of them (only affecting <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a>, for example on <a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a>) even critical because it could trigger "undefined behavior".Both bugs are regressions from new (performance) improvements added, one from trying to queue as many writes as possible when sending HTTP responses, one from using kqueue to provide timers and signals.See release notes for 0.8. Don't use 0.7. Sorry 🤪 <a href="https://github.com/Zirias/swad" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/swad</a>

Felix Palmen :freebsd: :c64:Now that <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> 0.7 is released, it's time to prepare a new release of <a href="https://mastodon.bsd.cafe/tags/poser" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poser</a>, my own lib supporting <a href="https://mastodon.bsd.cafe/tags/services" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#services</a> on <a href="https://mastodon.bsd.cafe/tags/POSIX" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#POSIX</a> systems, following a <a href="https://mastodon.bsd.cafe/tags/reactor" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#reactor</a> with <a href="https://mastodon.bsd.cafe/tags/threadpool" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#threadpool</a> design.During development of swad, I moved poser from using strictly only POSIX APIs (with the scalability limits of e.g. <a href="https://mastodon.bsd.cafe/tags/select" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#select</a>) to auto-detected support for <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a>, <a href="https://mastodon.bsd.cafe/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a>, <a href="https://mastodon.bsd.cafe/tags/eventports" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#eventports</a>, <a href="https://mastodon.bsd.cafe/tags/signalfd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#signalfd</a> and <a href="https://mastodon.bsd.cafe/tags/timerfd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#timerfd</a> (so now it could, in theory(!), "compete" with e.g. libevent). I also fixed quite some hidden bugs, and added more base functionality, like a <a href="https://mastodon.bsd.cafe/tags/dictionary" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#dictionary</a> using nested hashtables internally, or <a href="https://mastodon.bsd.cafe/tags/async" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#async</a> tasks mimicking the async/await pattern known from e.g, <a href="https://mastodon.bsd.cafe/tags/csharp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#csharp</a>. I also deprecated two features, the periodic and global "service tick" (superseded by individual timers) and the "resolve hosts" property of a "connection" (superseded by a separate resolve class).I'll have to decide on a few things, e.g. whether I'll remove the deprecated stuff immediately and bump the major version of the "posercore" lib. I guess I'll do just that. I'd also like to add all the web-specific stuff (http 1.0/1.1 server) that's currently part of the swad code as a "poserweb" lib. This would get a major version of 0, indicating a generally unstable API/ABI as of now....And then, I'd have to decide where certain utility classes belong to. The rate limiter is probably useful for things other than web, so it should probably go to core. What about url encoding/decoding, for example? 🤔Stay tuned, something will come here, maybe helping you to write a nice service in plain <a href="https://mastodon.bsd.cafe/tags/C" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#C</a> 😎:<a href="https://github.com/Zirias/poser" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/poser</a>

Felix Palmen :freebsd: :c64:Just released: <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> 0.7! 😎 Swad is the "Simple Web Authentication Daemon". If you're looking for a solution to add cookie/form <a href="https://mastodon.bsd.cafe/tags/authentication" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#authentication</a> to your <a href="https://mastodon.bsd.cafe/tags/nginx" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#nginx</a> reverse proxy, or maybe even a <a href="https://mastodon.bsd.cafe/tags/lightweight" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#lightweight</a> alternative to <a href="https://mastodon.bsd.cafe/tags/Anubis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Anubis</a> which leaves the actual proxying to nginx, this might be for you! It is designed for use with nginx' auth_request, written in pure C, with minimal dependencies (zlib and, depending on build options, openssl/libressl and/or libpam), and compiles to a small binary (currently between 150kiB and less than 300kiB depending on compiler and target platform).Swad should work on many <a href="https://mastodon.bsd.cafe/tags/posix" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#posix</a> (and almost) systems. It's actually tested on <a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a> (in "production" use, but on a very low-traffic private site), and quick functionality tests also done on <a href="https://mastodon.bsd.cafe/tags/Debian" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Debian</a> (<a href="https://mastodon.bsd.cafe/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a>) and <a href="https://mastodon.bsd.cafe/tags/OpenIndiana" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenIndiana</a> (<a href="https://mastodon.bsd.cafe/tags/Illumos" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Illumos</a>, open-source <a href="https://mastodon.bsd.cafe/tags/Solaris" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Solaris</a> descendant).As announced, this release doesn't bring any new features (in terms of WHAT it can do), but great improvements "under the hood", that should help performance at least on some platforms, see release notes for swad 0.7.Read more, and download the .tar.xz (to build and install it 😆) here: <a href="https://github.com/Zirias/swad" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/swad</a>

Felix Palmen :freebsd: :c64:The next release of <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> will probably bring not a single new feature, but focus on improvements, especially regarding <a href="https://mastodon.bsd.cafe/tags/performance" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#performance</a>. Support for using <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a> (<a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a> et al) to handle <a href="https://mastodon.bsd.cafe/tags/signals" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#signals</a> is a part of it (which is done and works). Still unsure whether I'll also add support for <a href="https://mastodon.bsd.cafe/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a>' <a href="https://mastodon.bsd.cafe/tags/signalfd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#signalfd</a>. Using kqueue also as a better backend for <a href="https://mastodon.bsd.cafe/tags/timers" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#timers</a> is on the list.Another hopefully quite relevant change is here:<a href="https://github.com/Zirias/poser/commit/798f23547295f89fa0c751f0e707c3474b5c689c" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/poser/commit/798f23547295f89fa0c751f0e707c3474b5c689c</a>In short, so far my <a href="https://mastodon.bsd.cafe/tags/poser" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poser</a> lib was always awaiting readiness notification (from kqueue, or <a href="https://mastodon.bsd.cafe/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a> on Linux, or select/poll for other platforms) before doing any read or write on a socket. This is the ideal approach for reads, because in the common case, a socket is NOT ready for reading ... our kernel must have received something from the remote end first. But for writes, it's not so ideal. The common case is that a socket IS ready to write (because there's space left in the kernel's send buffers). So, just try it, and only register for notifications if it ever fails, makes more sense. Avoids pointless waiting and pointless events, and e.g. with epoll, even unnecessary syscalls. 😉

Felix Palmen :freebsd: :c64:When writing a <a href="https://mastodon.bsd.cafe/tags/daemon" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#daemon</a> that follows best practices (handling of <a href="https://mastodon.bsd.cafe/tags/detaching" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#detaching</a> with a locked <a href="https://mastodon.bsd.cafe/tags/pidfile" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pidfile</a>, and <a href="https://mastodon.bsd.cafe/tags/SIGHUP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SIGHUP</a> for <a href="https://mastodon.bsd.cafe/tags/configuration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#configuration</a> <a href="https://mastodon.bsd.cafe/tags/reload" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#reload</a>), an extremely simple "init script" will do (reliably!) for <a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a>'s mewburn-rc. 😎 <a href="https://mastodon.bsd.cafe/tags/C" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#C</a> <a href="https://mastodon.bsd.cafe/tags/coding" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#coding</a> <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a>

Felix Palmen :freebsd: :c64:I'm trying to add "genric" <a href="https://mastodon.bsd.cafe/tags/signal" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#signal</a> handling to <a href="https://mastodon.bsd.cafe/tags/poser" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poser</a>. Ultimate goal is to provide a way for <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> to handle <a href="https://mastodon.bsd.cafe/tags/SIGHUP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SIGHUP</a>, although signal handling must be done in poser's main event loop (signals are only ever unblocked while waiting for file descriptor events).Okay, I could just add explicit handling for SIGHUP. But a generic solution would be nicer. Just for example, a consumer might be interested in <a href="https://mastodon.bsd.cafe/tags/SIGINFO" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SIGINFO</a> which doesn't even exist on all platforms ... 🤔 Now, <a href="https://mastodon.bsd.cafe/tags/POSIX" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#POSIX</a> specs basically just say signal constants are "integer values". Not too helpful here. Is it safe to assume an upper bound for signal numbers on "real world" OS implementations, e.g. 64 like on <a href="https://mastodon.bsd.cafe/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a>? Should I check <a href="https://mastodon.bsd.cafe/tags/NSIG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NSIG</a> and, if not defined, just define it to 64? 🙈 <a href="https://mastodon.bsd.cafe/tags/C" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#C</a> <a href="https://mastodon.bsd.cafe/tags/coding" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#coding</a> <a href="https://mastodon.bsd.cafe/tags/question" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#question</a>

Recent searches

Search options

Administered by:

Server stats: