Several months have passed since the last EndBASIC release and, since then, you might have seen me talk nonstop about an “EndBASIC service” in social media… which has sounded like vaporware. That changes today.

After about four months of work, I am ecstatic to announce that EndBASIC 0.7 is here. And these haven’t been four months of idle time. No, no, no. It has taken four months of my scarce free time to deliver this because 0.7 is a humongous release on various fronts.

Here are the highlights:

📀 Hello, drives. The file system exposed by EndBASIC now supports drives. This feature has an MS-DOS feeling to it, which is intentional in keeping with the retro looks. And the reason I built support for drives is because of…
⛅️ Hello, cloud. The largest chunk of work in this release has gone into the making of a cloud service to support file sharing. From within EndBASIC, you can now type LOGIN to log into your account and access your own CLOUD:/ drive. From there you can SHARE your own files or access files shared by other people with commands like MOUNT "X", "cloud://cool-user".
🕸 Hello, website. Every time I’ve had to quote EndBASIC in random conversations, I’ve missed having a good landing page. The README.md on GitHub has more-or-less done the trick until now, but it has become insufficient. So with this release, I’ve built a website. The website is an important piece of the cloud puzzle to guide you through the features and the account creation process.

Keep reading if you want the insider view of this release. Otherwise, if you just want to play with it, head to the new shiny landing page and enjoy as much as I have building this:

https://www.endbasic.dev/

WARNING: Expect bugs though, both in the client and in the service. This is the first cloud service I ship built on Azure so there are known and unknown rough edges. Do not hesitate to file bugs, and please contact me with any feedback!

Now, the insiders story.

Why “the cloud”?

The idea behind the EndBASIC cloud service started boiling after someone posted an innocent reply to one of my previous announcements asking “how can we share our creations with others?” Being EndBASIC an app that runs on the web, my instant thought was “well, those files should just be shareable from there instead of being stuck on your browser’s local storage, shouldn’t they?”

I procrastinated on that thought because it seemed hard. The funny this is that, even though I work for a cloud provider now and have worked for another one in the past, I had never written a cloud-based app myself. With a lot of unknowns, I was quite hesitant to embark on this quest. This was, however, the perfect opportunity to fight those demons and, as you may recall from the Why am I building EndBASIC? post, a personal goal behind this project is to learn new stuff. So here we are.

That said, I have to confess that building the service has been exciting but also a lengthy and frustrating process. Figuring out OAuth and how to leverage Azure Active Directory for my needs took a couple of weekends of dedicated time. After that, I had a prototype of the service running in just a few hours. But turning that prototype into a piece of code I was proud enough to ship has taken… all those months.

Service design

My goals behind the design of the cloud service were simple:

Azure. I work for Azure so I wanted to learn about our product from the client side. Working on this project has taught me a few things that have already helped at work.
Rust only. EndBASIC is written in Rust and I wanted to do the same for the cloud service. From past experience writing a trivial REST server, it seemed like a good fit—and I really wanted to play with a real service written in Rust. Of course, this made things more complicated than they could have been because Azure does not have a first-party Rust SDK and the unofficial one is still very rudimentary.
Minimal cost. This is a personal project and I want to keep my costs as close to zero as possible because I do not want to have the looming thought of “this isn’t working; I have to turn it down to not waste money”.
Minimal lock-in. The cloud offers tons of features, and while I wanted to leverage some (such as Azure Functions to not deal with provisioning servers manually), I did not want to have hard dependencies on the cloud. I want the ability to move out.

Database difficulties

A huge time sink in the building of the cloud service went into interacting with the backing database.

You see, when I started looking at hosted database options in Azure, my first thought was to go with PostgreSQL. But the provisioning tool claimed that I’d be spending over $200 a month for the service—which is just insane for my rudimentary needs. I knew there was a serverless option that only charged for usage but that option is only available for Microsoft SQL Server (MSSQL) as far as I can tell.

“OK, fine, a database is a database, and I need no fancy features from them, so MSSQL can work as well as PostgreSQL.” I thought. Well, yes, except for client-side issues. For my server, I chose to use the sqlx crate as it had good reviews and an MSSQL connector. But, at first, I couldn’t even get a trivial connection to work. Soon enough, I found out that the MSSQL serverless option requires TLS and that sqlx did not implement TLS for MSSQL.

Bummer. But how hard could it be to fix this? The sqlx code to handle TLS in the PostgreSQL and MySQL connectors looked very simple, so replicating it for MSSQL seemed easy too. Ha, ha, ha. It was indeed much harder to build and explains why it hadn’t been done yet. I wasted a couple of weeks reading TDS and TLS protocol documentation, fighting with Wireguard, and doing tons of tests for this patch to sqlx to work—until the stars aligned and I got a successful connection with PR 1200 and PR 1203. I think I need to come back to this in a separate post…

Anyhow, all good, right? Well… not so fast. The more I kept building my service, the more deficiencies I kept encountering in the sqlx MSSQL client. Missing data types and incorrect handling of large query payloads turned out to be very crippling, and the last straw was finding out that I had to limit EndBASIC file sizes to less than 1KB for the sqlx connection to remain stable. Unacceptable, so I had to abandon this connector. (By the way: this connector is also going proprietary in the next sqlx release. I suspect that the proprietary version will be much better, but that was one more reason to avoid it.)

This made me go back to the database creation tool in Azure and look at PostgreSQL again. And by massively reducing the hardware requirements of the server (an option that was hidden and somehow discouraged), I could bring down the monthly costs to $15-20. Much more reasonable. After all, this basic server still has 2 CPUs and loads of RAM, and those hardware requirements can go a long way. Which makes me wonder: how much money are people paying by choosing the defaults and ending up with overprovisioned servers they may never benefit from? We used to build highly performing services with machines as powerful as this in the past!

With that cleared, I moved to PostgreSQL, a choice that better aligns with the “minimal lock-in” principle. Luckily, I had already built a database abstraction in my code so that I could run all tests against SQLite instead of MSSQL, so rewriting the production variant to talk to PostgreSQL instead of MSSQL was trivial.

Security and privacy difficulties

Security is hard. You would hope that using cloud solutions would make dealing with security simple and safe. And that’s partially true: if you follow the tutorials to the letter, you’ll probably get authentication working correctly and safely. But there are lots of knobs and little details in the process that, if implemented incorrectly, could lead to security problems. Furthermore, and because there is no good SDK for Azure for Rust yet, I ended up having to deal with hand-crafted API calls… so that’s one more place where things can go wrong.

What about authorization, though? The EndBASIC cloud service is all about file sharing, so the authorization checks belong in my service’s logic. That’s another fragile part where subtle bugs can result in disasters. Hopefully the service is pretty simple, but still, it’s an area that’s uncomfortable to deal with.

And what about unintentional information disclosure? This is where things get very subtle. For example, people shouldn’t be allowed to discover whether someone has an account by looking at different error messages in API calls. But it’s very easy to leak these details if you haven’t thought about them. Many of these scenarios crossed my mind, and I think I have done a reasonable job in these areas, but again, these are subtle issues and I’m quite positive I’ve missed some.

And, lastly, abuse. What could possibly go wrong with offering a free storage cloud service? Uh… a lot, and remember that I want to keep costs minimal. Thankfully, the files you might create from EndBASIC are going to be very small, so I ended up building per-user disk quotas and setting those to extremely low values by default. You’ll get a whopping 64KB of disk space on your cloud drive, very in line with the retro feeling! But this was yet another delay in shipping this project.

Parting words

During these few months, I have learned quite a few new things that have unblocked long-standing mental blocks around cloud services. As mentioned earlier, OAuth has been one of them. And while I still have a lot of unknowns in that specific area, the knowledge I’ve gained has already helped me at work to diagnose real customer issues.

I have also built a bunch of components that will be useful in the future. For example, I spent much more time than I should have building a database abstraction that supports SQLite for unit testing instead of the production PostgreSQL, and I also spent a lot of time building a logger implementation that records messages into the database (instead of stderr). You can say that these were unnecessary for an MVP, but these components and patterns will come in handy for any other service I want to build in the future—so I had an incentive (and no time pressure) to do the right thing.

In the end, this was all about gaining experience points and having fun along the way—and so far, I think it paid off. There is a lot more to say, but I’ll save those topics for other posts.