Senko Rašić

A practitioning bit-shifting magician turned cat herder

Long time ago, information used to be hard to find. You had to go out there and look for it. Nowadays, a lot of information is at our fingertips at any moment.

Now we have the opposite problem: too much information. The challenge today is curation: we want to consume a reasonable amount of information that's relevant to us.

What's “reasonable” and what's “relevant” is tough to define as it is person-specific. What's reasonable and relevant to you may not be the same as what's reasonable and relevant to me.

From curation ...

Curation of information is nothing new: newspapers, radio and TV have done it since the beginning. We relied on the media to pick what's most important and only kept the choice of a source we trusted or liked.

As the Internet grew, the amount of information grew exponentially. One of the reasons for Google's early success is that it was very good at curating this information. While the other search engines could also find 10,000 pages matching your search, Google was the best at picking what was most useful, informative and relevant.

Other services were also trying their best at curation: Netflix had a great recommendation algorithm, and have for years organized a public competition to optimize it and provide even better recommendations.

.. to filter bubbles

Over the years, as the various algorithms got improved and optimized, they got subjectively worse for us, the users. Google is in arms-race with SEO folk, Facebook prefers to serve you content that will rile you up, Netflix' algorithm seems braindead and the less we talk about YouTube recommendations the better.

Why is this? Turns out that, at some point in their lifecycle, big companies had to choose between optimizing for the user experience and optimizing for revenue. The algorithms improve allright ... but using a different metric.

Google now hyper-optimizes for what it thinks you should see (filter bubble), Facebook serves you whatever will keep you on the site for longer (mind-numbing memes interspersed with viral outrages), and Netflix brings up and center its own content that it would like you to get hooked at (and not cancel the service), not the things you might actually want to see.

Curation for the benefit of the user, has turned to serving targeted content, for the benefit of the company.

Enshittification

Enshittification is an ugly word to describe an ugly thing. Basically, it is a strategy change in an online platform where it switches from user growth (where optimizing user experience is paramount) to revenue growth (where you need to squeeze maximum income from each user).

Enshittification, or platform decay, is not limited only to Internet media companies. But it is especially visible in online startups that grew big by giving away their product or undercharging for it, in order to grow as fast and big as they can.

At some point you got to earn money, and you have to earn as much money as possible, and since you have a mostly-captive audience (there is only one Google, Facebook, YouTube or Twitter), they won't leave if the user experience is marginally worse.

And so the UX frog gets slowly cooked. People today are hooked on Facebook scrolling, Twitter mob rages, memes everywhere and 10-second dopamine hits on TikTok or YouTube Shorts. It's the sugar thing all over again.

Information detox

Faced with this, some people try limiting their consumption of this kind of content (me included, see my Junk Social and Digital hygiene posts).

This can work if you're willing to limit your information access, don't suffer from the fear of missing out (FOMO) and have the mental strength to not succumb when you're tired, bored or just open up your favorite social media app on autopilot because it's on your mobile phone home screen.

But it's hard work: you're basically building and maintaining a Great Wall between yourself and most of the modern Internet.

Curation on our terms

Can we do it better? Is there a way to again outsource curation of content that is optimized for us, the users?

Curation is hard work, whether you want to build and maintain sophisticated algorithm or AI to do it, or if you have actual people doing the work. And it's not entirely obvious how that would work without degenerating into the filter bubbles and social media we have today – that's how some of those companies started, after all.

As a technologist I do believe a solution is within a realm of possiblity. As someone who's watched Internet grow from an academia and hobbyst garden into what it is today, I am skeptical we'll reach that solution (and if you really want to be scared about the prospects of it, go read The Master Switch).

A creative robot

Any sufficiently advanced technology is indistinguishable from magic.

Modern AIs and ChatGPT in particular look like magic to many people. This can lead to misunderstanding about their strengths and weaknesses, and a lot of unsubstantiated hype.

Learning about a technology is the best antidote. For curious computer scientists, software engineers and anyone else who isn't afraid of digging a bit deeper, I've compiled a list of useful resources on the topic.

This is basically my reading/watching list, organized from more fundamental or beginner friendly to the latest advances in the field. It's not exhaustive, but should give you (and me) enough knowledge to continue exploring and experimenting on your own.

General overviews

If you don't have a lot of time or don't know if you want to dedicate effort in learning the ins-and-outs of modern AIs, watch these first to give you a general overview:

Fundamentals of neural networks

The videos here provide both teorethical and hands-on introduction to the fundamentals of neural networks.

MIT Introduction to Deep Learning

A good theoretical intro is MIT's 6.S191 class lectures, especially the Introduction to Deep Learning and Recurrent Neural Networks, Transformers and Attention.

These overview lectures briefly introduce all the major elements and algorithms involved in creating and training neural networks. I don't think they works on their own (unless you're a student there, do all the in-class excercises, etc), but it's a good place to start with.

The topics discussed here will probably make your head spin and it won't be clear at all how to apply them in real life, but this will give you the lay of the land and prepare you for practical dive-in with, for example, Andrej's “Zero to Hero”.

The example code slides use TensorFlow. Since Andrej's course uses PyTorch, going through both sets of lectures will expose you to two most popular deep learning libraries.

Neural Networks: Zero to Hero

An awesome practical intro is the Neural Networks: Zero to Hero course by Andrej Karpathy (he also did the busy person's intro to LLMs linked above).

Andrej starts out slowly, by spelling out the computation involved in forward and backward passes of the neural network, and then gradually builds up to a single neuron, a single-layer network, multi-layer perceptron, deep networks and finally transformers (like GPT).

Throughout this, he introduces and uses tools like PyTorch (library for writing neural networks), and Jupyter Notebook, and Google Collab. Importantly, he first introduces and implements a concept manually, and only later switches to a PyTorch API that provides the same thing.

The only part where things look a bit rushed is the (currently) last – GPT. There's so much ground to cover there that Andrej skips over some parts (like the Adam optimization algorithm) and quickly goes over the others (self-attention, cross-attention).

Overall a great guide. You only need to know the basics of Python, not be afraid of math (the heaviest of which is matrix multiplication which is spelled out), and do the excercises (code along the videos) without skipping the videos that don't seem exciting.

Understanding Word2vec

Both the MIT and Andrej's lectures touch on embeddings (the way to turn words into numbers that a neural net can use) only lightly. To deepen your understanding, Illustrated word2vec article explains *word2vec+, a popular word embedding algorithm, step by step. It also features a video explanation for those that prefer it to text.

Another good lecture on the topic is Understanding Word2vec.

CNNs, autoencoders and GANs

The MIT lectures mention earlier also contain lessons on Convolutional Neural Networks, autoencoders and GANs, which are important building blocks in neural networks used in vision.

Again, these are high level overviews and although formulas are present, the lectures more give an overview of the algorithms without going into too much detail. That makes them ideal prequel to the Practical Deep Learning course by Fast.ai.

Diffusion models

Diffusion models build on top of CNNs to create image-generating and manipulating AI models. Beyond the general overview linked earlier, the Introduction to Diffusion Models for Machine Learning is a deep dive into exactly how they work.

Practical Deep Learning

Practical Deep Learning is a free course by Fast.ai that has (current count) 25 lectures covering both high-level practical parts of neural networks and the underlying fundamentals.

In particular in Part 2 they cover “zero to hero” on Stable Diffusion, a powerful image-generation AI model.

Large Language Models

These resources go in-depth about constructing and using large language models (like GPT):

Transformers

Andrej's course goes over the transformer (building blocks of GPT) architecture, but the complexity makes it easy to get lost at first pass. To solidify your understanding of the topic, these two are super useful:

The Illustrated Transformer describes the transformer (building blocks of GPT) in detail while avoiding tedious math or programming details. It provides a good intuition into what's going on (and there's even an accompanying video you may want to watch as a gentler intro).

Follow that up with The Annotated Transformer which describes the scientific paper that introduced Transformers and implements it in PyTorch. Since it's 1:1 annotation of the paper, you need a lot of understanding already so only attempt going through this once you've watched both Andrej's course and once you've read and understood the Illustrated Transformer.

Reinforcement Learning through Human Feedback

Language models are good at predicting and generating text, which is different from answering questions or having a conversation. RLHF is used to fine tune the models to be able to communicate in this way.

Illustrating Reinforcement Learning through Human Feedback from folks at HuggingFace (an open source AI community) provides a good overview of RLHF. They also did a webinar based on it (video is the complete webinar, link jumps directly to start of RLHF description) based on the blog post.

If you want to dive deeper, here's the InstructGPT paper from OpenAI, which basically describes the method they used to create ChatGPT out of GPT3 (InstructGPT was a research precursor to ChatGPT).

Fine-tuning

Fine-tuning allows us to refine or specialize an already (pre)-trained LLM to be better at a specific task (RLHF is one example).

Sebastian Rashka's Finetuning Large Language Models explains a few common aproaches to finetuning, with code examples using PyTorch. He follows that up with Understanding Parameter-Efficient LLM Finetuning, a blog post discussing ways to lower the number of parameters required, and an in-depth article about Parameter-Efficient LLM Finetuning with Low-Rank Adaptation (LoRA).

Full courses

If you want a really deep dive (undergrad or higher level), follow these courses including doing the excercises / playing around with the code:


This is a living document (ie. it's a work in progress and always will be). Come back in a few weeks and check if there's anything new.

ChatGPT and other AIs are all the rage and I see many (mostly junior) programmers worrying if the market for devs is going to dry up.

You don't need to worry, and here's why.

ChatGPT is scarily good. It really is. No it's not better at web search than Google (yet), and it's nowhere near being sentient. But for the tasks where it makes sense, it's very good. So I'm not going to tell you that you don't need to worry because it's a useless tool.

ChatGPT (and other AIs) is a tool, in the same way a calculator is a tool or a compiler is a tool. The word “calculator” used to refer to people doing number crunching. My first calculator was a handheld device. Now it's just an app. Yes, calculators, the people, lost their “job” doing mind-numbing number crunching, but they were able to work on more interesting problems in math, physics, or what have you.

Same with compilers. When compilers were invented, people were furious that someone thought a machine could do a better job than an expert programmer at crafting assembly code. Today almost nobody writes directly in assembly, except in rare cases where that makes sense. But “assembler programmers” didn't lose their jobs, they just became “C programmers” or “Lisp programmers”.

It is the same with the new AI models. They are very effective on a whole other level than just number crunching or compiling software, but at the end of the day they're just tools, like programming languages, libraries or APIs.

If you've been a programmer for more than a few years, you know you always need to learn new stuff and stay on top. You'll need to invest some time to learn what ChatGPT and others can do, or can't. What you don't want to do is completely ignore the trend, or (equally bad) think it'll solve all your problems (or put you out of business). And as John Carmack said in a recent tweet keep your eyes on the delivered value and don't over focus on the specifics of the tools.

I've been a programmer for some 30 years (20 or so professionally) and my usual reaction to new stuff is “oh, so they're reinventing that particular wheel again”. Yet the current crop of AIs gets me really excited, like I was a kid again uncovering the vast potential of what you can do with a machine that you can order around! I've been playing with ChatGPT and it makes me faster and it makes programming (more) fun!

Not by writing my code – I don't use it like that because it does produce bugs and it can hallucinate stuff. I use it to explore (how I might go about doing X), and to quickly recall something I forgot (how a particular API or library function is used, for example). As Simon Willison (of Django and Datasette fame) puts it AI-enhanced development makes me more ambitious with my projects.

This is just a beginning, and we're just seeing a boom in integrating these AIs into everything else. Copilot, Bing search, are the big names but people are experimenting with integrations with anything under the sun (I made a service that creates API backends based on your project description, for example). Time will tell which of these will be truly useful, but I have no doubt there will be a lot of them.

AI is a tool, with limitations, but with a lot of potential. It would be a shame not to use it effectively. It won't put you out of a job, it will make your job better.

As a long-time Hacker News reader, I've seen trends come and go. One of those trends is popularity of programming languages, manifested through the number of stories referencing a language.

Examples include “Building a Cache in Elixir”, “Helix: Neovim inspired editor, written in Rust” or “Text formatting in C++ using Libc++”.

Since these “in $LANGUAGE” articles come up fairly often, I was curious to see what we can glimpse about programming language popularity just by counting them. This is easy to do using HN Search API provided by Algolia.

Photo by Thomas Tastet @ Unsplash

Methodology

I used it to search for stories in the past year, where the phrase “in $LANGUAGE” appears in the story title. The search included C, C++, C#, Clojure, Dart, Elixir, Erlang, F#, Go, Haskell, Java, JavaScript, Kotlin, Lisp, Lua, PHP, Python, Ruby, Rust, Scala, Scheme, Swift, TypeScript, and Zig languages.

I also included Racket Scheme and added its numbers to Scheme. I'd have included Common Lisp as well, but there were no stories mentioning it explicitly in the title in the past year. Also, since C would match both C++ and C# results in this simple string search, I deduplicated the results to get the correct count.

I ranked the languages in three ways: by the number of stories, number of comments, and sum of points. The results were fairly consistent across all three metrics.

If you're interested in running this yourself, this is the script I used (you'll need the requests package to run it). It downloads and caches search results as JSON in /tmp/hn, sorts the languages, and prints the results.

I ran the script on October 12th, 2022.

And the winner is ...

... Rust, by a wide margin. Here are the full results:

By the number of stories:

  1. Rust (573 stories)
  2. Python (375 stories)
  3. Go (332 stories)
  4. JavaScript (200 stories)
  5. C (143 stories)
  6. C++ (136 stories)
  7. TypeScript (79 stories)
  8. Java (76 stories)
  9. Ruby (57 stories)
  10. Elixir (49 stories)
  11. Haskell (48 stories)
  12. Swift (41 stories)
  13. C# (32 stories)
  14. PHP (31 stories)
  15. Zig (27 stories)
  16. Clojure (23 stories)
  17. Scheme (13 stories)
  18. F# (12 stories)
  19. Kotlin (9 stories)
  20. Erlang (8 stories)
  21. Lisp (8 stories)
  22. Lua (8 stories)
  23. Dart (4 stories)
  24. Scala (4 stories)

By the number of comments:

  1. Rust (6384 comments)
  2. Go (3394 comments)
  3. C (3027 comments)
  4. Python (2783 comments)
  5. JavaScript (1583 comments)
  6. C++ (771 comments)
  7. Zig (735 comments)
  8. Java (615 comments)
  9. Ruby (396 comments)
  10. TypeScript (341 comments)
  11. Haskell (301 comments)
  12. PHP (223 comments)
  13. Scheme (177 comments)
  14. Swift (165 comments)
  15. C# (151 comments)
  16. Elixir (133 comments)
  17. Clojure (82 comments)
  18. Lua (75 comments)
  19. Lisp (53 comments)
  20. F# (45 comments)
  21. Erlang (39 comments)
  22. Kotlin (7 comments)
  23. Dart (1 comments)
  24. Scala (0 comments)

By points:

  1. Rust (16887 points)
  2. Go (7144 points)
  3. Python (7048 points)
  4. C (5422 points)
  5. JavaScript (3644 points)
  6. Zig (2427 points)
  7. C++ (1846 points)
  8. Java (1595 points)
  9. Ruby (1044 points)
  10. TypeScript (998 points)
  11. Haskell (846 points)
  12. Elixir (751 points)
  13. Scheme (549 points)
  14. Swift (447 points)
  15. PHP (402 points)
  16. Clojure (365 points)
  17. C# (334 points)
  18. F# (331 points)
  19. Erlang (322 points)
  20. Lua (238 points)
  21. Lisp (154 points)
  22. Kotlin (34 points)
  23. Dart (14 points)
  24. Scala (7 points)

Hacker News loves Rust

Go, Python and JavaScript are quite popular, but Rust absolutely blows the competition out of the water. It had almost as much stories and comments as the languages in second and third place combined, and was awarded even more points.

If you're a regular HN reader, this isn't surprising. As a modern language with passionate community, Rust is invariably mentioned in most conversations about languages in general, and stories about using Rust for a new project or in a new are always exciting.

And I'm not even counting the “Rust in the Linux kernel” theme, which alone got around 600 points across various stories in the past year.

The new kid on the block is ... Zig

What did surprise me is the relative popularity of Zig, a fairly young language with still a relatively small comunities – I'll be sure to keep an eye on it.

(comment on Hacker News) | (follow me on Twitter)

As a software developer with many years of remote and in-office experience, I am excited by the current shift that's making remote and hybrid work commonplace.

The pandemic was a global tragedy and a big disruptor to every aspect of the lives of people everywhere. One of the few silver linings was that it forced a rethinking of the importance of location for the knowledge worker and what “office” actually means.

In my two decades of professional experience, I spent about half of it working remotely. The other half we had a small office (10 to 15 people), but mostly worked with clients overseas. In my time as both an individual contributor, and manager of dev teams, I had to handle the challenges arising from, and enjoy the advantages of, remote work.

a digital nomad in its natural habitat

My view is naturally colored by my experience. I am a software engineer and have worked in IT my entire career. While I believe most of what I say here can be generalized, there are nuances for every profession.

Location, location, location

Let's get obvious out of the way. Not all work is – or will be – possible to do remotely.

Although technology will continue to push the limits (remote truck driving, anyone?), many jobs, from surgeons to shopkeepers to street sweepers, still require people to be in a certain location to do their job. This does not mean those jobs are lesser, or that they will lose out in the “remote work revolution” – quite the opposite. Fewer people commuting to work and crowding in hot locations means less traffic, less stress and more room for those that choose to – or have to – be there.

Bare necessities

My job as a software developer consists mostly of reading from a computer screen and typing on a keyboard, with the occasional use of a mouse. I can do that anywhere on the planet, provided there is electricity and an adequate internet connection. In reality, what I consume (as a work input) and produce (work output) is information, and that's easily sent across the world.

The same holds mostly true for most knowledge workers. When push came to shove (the pandemic lockdowns), a lot of people realized the work they do can be performed remotely.

Hierarchy of needs

Of course, that's not all there is. We still need information, often implicit, from our coworkers, about the problem we're solving, and the wider context our company, customers, and clients are operating on. Much of this is recorded poorly, or not at all. Remote interactions are harder than face-to-face and Zoom fatigue is real. And sometimes we need specialized office or lab equipment.

And as much as we love working in our pajamas, we might miss office chit-chat or getting that energized feeling from being a member of a well-oiled team of awesome people doing great work together.

Back to life, back to reality, back to the here and now

Now that things are normalizing a bit (I know it's a stretch to use that word in the middle of 2022), companies are starting to mandate back-to-office or hybrid work. Conspiracy theories about reasons are many, from wanting to micromanage workers better, to making sure expensive office leases are utilized, to being a sneaky way to lay people off without actually giving them the pink slip.

However this is resolved, the genie is out of the bottle. Remote work has previously been regarded as a unique perk or very specific work arrangement. Now, wherever it's allowed or not in a particular company, it's normal. I think that's a good thing.

Checking our privilege

Us IT folks have it easy. The job market is hot and there are plenty of opportunities for good engineers. Yes, there are hiring freezes and some companies are laying off people. In most of the cases I heard of, people were quickly snatched by the competitors.

This puts us in a position to demand the ability to work remotely, and if needed, to quit and join a company that will allow that. Not everyone is so lucky and there are legions of knowledge workers who have returned (or will have to return) to the office just because the bosses decreed it.

In IT, and especially in software development, the shift is real, and companies need to adapt.

In-office vs remote work

There is a fundamental divide between in-office and remote work, and that is how the information within the company flows. Companies where in-office work is the norm can get away with much larger implicit context and rely on employees communicating directly ad-hoc, as needed.

To take full advantage, such companies maximize the overlap between the employees' working hours (9 to 5 everyone!), and try to cluster employees together (if not all the company can be in the same location, at least the teams working together) and design office space to optimize for interaction (leading to unfortunate things like open offices).

Remote-first companies focus on async work (work on your own time, communicate asynchronously with coworkers) making it possible to hire globally. Instead of spending on office space, they give workers budgets to improve their workspace or equipment. But most of all, remote-first companies take communication at all levels seriously. From office banter to all-hands meetings, stuff is happening online and is often saved or recorded for the benefit of people who are not online at that very moment.

It's easy to contrast these two extremes, but many companies will probably lie somewhere in between.

Old-style hybrid work

“Hybrid” is a word that describes two different work arrangements, depending on where the company started from.

For traditional office-based companies that are introducing hybrid work, this usually amounts to a perk allowing employees to work from home a few days per week. Often, at least two days are fixed for meetings and any other work that must be performed on-site. This type of hybrid office can easily slip back into the “office is king” mentality, where the important things still happen in the office, not everything is shared with remote workers (not necessarily due to any malicious intent), and the more you're present, the faster you'll career trajectory will be.

This was mostly the status quo for remote workers before the pandemic. Except in rare enlightened companies, remote workers were de-facto second-class citizens. The more a company now stresses that being in-office is important, the more likely it'll fall back on this pre-pandemic default.

I believe that companies that are now requiring employees to be present at least three days a week will most likely fall back to this mentality.

If they combine this with hot-desking, where employees must find themselves a (possibly different) desk each day, it will only make things worse. I fear this may be a real possibility for many, to try to have the workers back but still save on office space.

Remote-first hybrid work

Another type of hybrid-location company is remote-first which recognizes there is benefit in getting people together or making it easy for people who want to work from the office to do so. These companies understand office is just another location. Hopefully, the environment is optimized for work (quiet environment, good equipment, etc) but in terms of work communication, it holds no special place.

A company I currently work with has such an approach. They're NYC-based, but have employees throughout the US, as well as people in Latin America and Europe. The main communication channels are all digital – Slack, Zoom, Notion, and email. Scheduling Zoom meetings across time zones is a bit challenging, but they manage and are looking to improve as they grow.

From remote-as-perk to remote-first hybrid

There's a tremendous opportunity for companies that are willing to switch from the office-based (or remote-as-perk a few days per week) model to the remote-first hybrid model. They can still make use of their existing office space (possibly downsized to reap financial benefits), reconfiguring it so it is support for people who want to work from the office, or who need to come to the office.

Essentially, they can turn their offices into a sort of internal coworking space. Employees that don't want or can't work from home can still come to the office. Coworkers or entire teams can independently or ad-hoc agree to come to the office for a day or a week for brainstorming sessions, project kick-offs, or just to socialize from time to time.

Remote-first, but not remote-always

In a few places I worked as a remote developer, companies had policies to bring everyone on the team together for a few days. This was sometimes combined with everyone going to a conference but hanging out in the off-times, or for everyone to be in the same place to start a new initiative. This was always a fun experience (even when paired with hard work) and I felt energized and ready to take on new challenges.

I believe fully remote companies should try to bring their people together at least once or twice per year. This can be company-wide (for smaller organizations) or per-team (easier to do with larger companies). If the company is fully remote and doesn't have an HQ, doesn't matter: pick a nice location and if there's a conference or another interesting event, that's even better.

Remote vs overseas

A lot of companies these days say they hire remotely but within a country. For example, a job listing may say “remote, US-only”.

This is understandable: it's legally easier to hire within the country you're incorporated in, the cultural differences are smaller, and time zone difference is more manageable. it can also be harder to find remote talent since you don't know where or how to look. The stigma of hiring overseas developers as “outsourcing to cheap labor” doesn't help.

Cultural differences are real and should not be ignored. Being cognizant of the different ways people communicate (or avoid difficult subjects) is important. I've found that (respectfully) overcommunicating helps, as it reduces the potential for confusion. Time zone differences can have a larger or smaller impact, depending on whether your team's communication is more synchronous or async. If you have daily Zoom meetings and your team is spread over Australia, Europe, and the US, you'll have a lot of unhappy people. Somewhat localizing the teams, and making most communication asynchronous help.

Remote doesn't mean cheap labor

If you pay peanuts, you'll get (code) monkeys. Good developers anywhere know what they're worth. That said, the differences in cost of living are huge, and people do take into account the (in)convenience of living anywhere. $100K means a very different lifestyle to someone in San Francisco than to someone in Vienna or Manila.

Developers are ambivalent toward location-based salaries – on the one hand, we'd prefer to be paid what we're worth (based on the value we provide), not based on where we live. On the other hand, we don't accept that there are talented and skilled people that can provide the same value for a slightly (or more than slightly) lower price, in lower-income countries.

If companies were completely oblivious to someone's location, it would be natural for them to want to pay a lower price for the same value, so they'd hire in lower-income locations (within a country or overseas). But they're not. They do prefer local (or nearby) employees, and they are willing to pay a higher price for that. A direct consequence of this is that at least some portion of the salary is going to be location-based.

As knowledge workers, we should get to terms with that fact. We are paid based on the value we provide, but not in isolation, and the job market will always have an impact.

The future is already here – it's just not evenly distributed

The pandemic turned remote work from a fringe benefit to a normal, accepted, way to do your job, in cases, companies and jobs where that makes sense. The technology will keep pushing the boundaries where that's possible, and the knowledge workers will keep pushing to have that option.

Companies will have to adapt: by doubling down on office (and paying extra for it), embracing the new opportunity to broaden the talent pool, or finding a hybrid sweet spot that makes sense for both them and their employees.

There's no going back.

Last month, after successful transition of our users to Miro, I shut down the public servers of A Web Whiteboard (AWW App), marking the end chapter of a decade long story.

Throughout, millions of users have collectively spent a millennia drawing, learning, collaborating, and just having fun with AWW. It's been used in schools, FAANGs and everywhere in between. Early in 2021 we joined forces with Miro, the world's leading online whiteboard platform.

The start of the story is much humbler. I never intended or even imagined it would be a success it has been.

Origins

Ten years ago, back in 2011, netbooks were all the craze. What started some six years earlier with EeePC turned into an entire product category with Cambrian explosion of options. I got my hands on one with a curious feature – a touchscreen. At the time, system UI wasn't at all suited for clumsy fingers instead of precision controlled mouse pointer.

So there I was, back in 2011, trying to find a simple, nice, drawing program for Linux that would work well, UI-wise, with touchscreens. I didn't find any so decided to write my own. I took this as an opportunity to learn HTML <canvas> and write it as a web app.

It was all pretty rudimentary. You had a pencil in a few colors, eraser, and ability to clean the entire canvas. This first version even lacked the ability to save the result – you could always screenshot the page if you wanted to save.

This wasn't even a side project at the time – just a fun experiment. For kicks, I decided to put it up online and, after thinking for a whole two seconds, named it A Web Whiteboard. Since aww.com was taken, I used awwapp.com for the domain (nameapp.com was a popular way of getting around the domain name squatters at the time).

Sharing

A few months in, someone mentioned it'd be cool if you could actually have multiple people drawing at the same time. I took the suggestion an opportunity to again dabble in new tech. At the time, websockets were just being entering mainstream use and Node was starting its meteoric rise, so I selected it for the synchronization backend and now all-but-forgotten RPC-over-sockets library called nowjs. Being primarily a Python developer, I kept to at least something familiar, using Flask for saving the board contents.

This is how it looked like:

First public version

I spread the word around, got some mention in the press and received positive feedback, including on Hacker News and Reddit. There weren't many users back then (the whole thing comfortably fit in RAM of a rather small VPS), but each time users gathered to draw together, or shared a saved drawing on social media, more people heard about AWW.

After the launch, I maintained AWW but treated it as a side hobby, and for a couple of years most of my focus was on other things.

Embedding

One thing I did from the start is organize AWW as whiteboard widget that could be embedded into other sites, with awwapp.com being just one of the sites that embedded it.

When I started getting requests to let people add it to their sites, it was easy to do. I set up a PayPal subscribe button and wrote up a simple doc on how to embed the widget. When someone subscribed, I'd add them to the accounts “database” (actually a JSON file) and gave them an unique API key to embed with.

To add whiteboard to their sites, my customers would include jQuery and add the following snippet:

<div id="wrapper" style="width: 800px; height: 600px;"></div>
<script src="http://static.awwapp.com/plugin/1.0/aww.min.js"></script>
<script>
    $('#wrapper').awwCanvas({apiKey: YOUR_API_KEY});
</script>

After a time, this netted me a few hundred bucks a month, which was nice but nothing I'd base my business on – or so I thought.

I also started getting more and more serious questions about using the whiteboard on awwapp.com for business, tutoring, etc, as well as feature requests. Most of my customers were tutors using it together with Skype to provide online lessons to their students. They often didn't have a site to embed the whiteboard on, were happy to use awwapp.com version and willing to pay for it.

The rewrite

As I started treating AWW as a viable business in its own right, I started thinking about rewriting it.

The old JSON file approach was getting unwieldy, as was the fact that all active board were being kept in memory, limiting growth. Finally, the old nowjs framework was unmaintained meaning we were stuck on an extremely old Node version (0.6 if I remember correctly).

I decided on a full rewrite with redesigned architecture and using modern (for the time!) JavaScript features – stuff like promises, the socket.io library and using code bundler tools. For database we selected MongoDB, primarily for ease of use and ease of clustering/failover. We also switched from a single server (and thus single-point-of-failure) to a load-balanced system with full failover (using haproxy for web server load balancing).

This was an explosion in ops complexity. Mostly warranted, but I definitely over-engineered some things, and hit many of the novice mistakes when scaling out.

We started the rewrite in the summer of 2014 and deployed the new version in spring of 2015. I say “we” because I had help – this was now a proper project internal to my dev agency with more people involved. However it still wasn't a priority and was done at off-times and between client projects.

The rewrite looked similar to the original

With the new version, we were finally able to sign up users directly on awwapp.com, again using PayPal subscriptions. We followed the freemium model, in which most of the functionality was free, but you could pay for some additionally useful features.

Ease of use

Throughout entire lifetime of AWW, I had one major focus, sometimes to the detriment of other aspects of the app: it had to be easy to use. Really easy to use. So easy a child could use it.

We did that successfully – in fact, many children did use it in kindergarten and preschool. It was great seeing pictures teachers shared of kids doing a collaborative class, or exploring maths using our whiteboard. “This is why I'm doing this”, I'd say.

For example, to start drawing on AWW you only had to click one button, labelled Start drawing, and less than a second later, you're drawing. And the only reason there was some text and a button to click first, was so people coming first time to AWW knew what the site was about! Before this, people coming up on an empty board were sometimes confusing what that was for.

Word of mouth

One thing we didn't do (except in a small experiment at the start with StumbleUpon) until much later, is promote or advertise AWW. In fact, I did hardly any outreach at all.

Virtually all of the (slow and steady) growth was entirely through word of mouth – our users telling others about us. This was helped by the fact that AWW is a collaboration platform, so users naturally wanted to share the drawing activity or results with others.

Combined with the ease of use and the fact most of the features were free, this made AWW a favorite whiteboard for many teachers and tutors. They were telling each other about it, featuring it in teacher seminars and doing video tutorials on it! This showed me how important and useful AWW was for a growing number of people and really warmed my heart.

Spinning off into a separate startup

Usage slowly but steadily grew over the years, to the point that there was simply too much to do on maintenance, customer support, inbound sales and further development, to treat AWW as a side-project.

I also realized that, as a technical person, I should get new people on board that will have “skin in the game”, a fresh perspective and complementary set of skills such as business development, sales.

I met Zvonimir at local startup event where he was just describing the challenges of his previous startup attempt. Zvonimir had tech chops and biz acumen in equal amount, combined with startup experience, but what I most liked was the levelheadedness when thinking about the realities of building (and selling!) a product in an uncertain environment.

We hit it off immediately and after some time I managed to convince him to join me as a cofounder. I consider this to be the best decision (and best sale!) I made throughout the development on AWW.

We spun off AWW as a separate startup in late 2016, incorporated it in the US using Stripe Atlas, brought several more people on board (a developer, a designer, and a marketing/support person) and set to work. We had three priorities:

The plan

First, to figure out who our users were, why they used us and what we can do to make us more useful to them. This involved identifying different markets we were playing in, such as EdTech, enterprise collaboration, etc. Up until that point we “lucked out” in that our product was useful – now we set to deliberately discover our customers and optimize our approach (feature set and messaging) towards them.

Secondly, to improve our user experience. AWW was simple and easy to use, but not so simple to figure out all the capabilities it had, or indeed why you'd want to pay for it. The more advanced “premium” functionality was hidden away unless you were a premium user, so many of our free users didn't even realize there was more to it.

The new UI

Also, as our features and supported use cases grew (and were expected to grow much further), bolting them onto the old interface made it more and more clunky over time. We set out to rethink the UI from scratch, still respecting our old imperatives (simplicity) but incorporating the lessons learned.

Third priority was to start working on our marketing, sales and business development, and to improve our customer support.

Up until then, customer support meant me answering email questions with a few days' delay. We aimed to lower the reply time to within a business day and to be active on more channels, such as Twitter and Facebook.

The online presence tied in to marketing. We didn't do any advertising, aside from a few experiments to test the waters and measure potential interest for various whiteboarding use cases. However, we did want to be present on Twitter, Facebook, LinkedIn, reaching out to blogs reviewing us, talk with teachers recommending us in workshops, and this tied in nicely with the customer relations part of the plan.

The funding

Now we were in business. We had several people and everyone worked exclusively on AWW, except me. I spent half my time on AWW and half managing my software development agency GoodCode.

Even though Croatian salaries are nowhere near US (and especially Silicon Valley) ones, having more people still meant more expenses. What was previously a profitable side project now turned into a startup that wasn't yet ramen profitable. I was funding the startup from the profits from my agency so we didn't need to worry about runway for the time being, but were still open to investments.

Our rough plan was to ramp up the product and get to profitability within twelwe months. Of course we hoped for much more – our valiant efforts will result in a hockey-stick growth and back to black in three months and then sky's the limit!

That, of course, didn't happen.

YC School

At that time we applied to several accelerators, including YC, but didn't get in. Though one of the reasons was for funding, our primary reason was the advice, learning experience, networking and fresh ideas.

We also applied to the first YC Startup School in 2017 and did get in! The online MOOC-style course combining online video lectures, weekly check-ins, discussions with mentors and our fellow founders were phenomenal. I imagine the intensity is nowhere near a “full” YC experience, but it was tremendously helpful for us.

YC Startup School Completion Certificate

First, the weekly check-ins did wonders for our accountability. We had metrics before and we did look at what we've achieved (or not) previously, but not in as critical and clear way we had to for Startup School. We soon found out our old non-actionable vanity metrics were pretty much worthless and that we didn't even have an easy way to get to the right numbers. Focusing on the real was like wiping your glasses (or the window) and actually seeing what was going on.

Weekly discussions were as valuable. We received (and gave out) thoughtful and probing questions which made us reflect on our implicit assumptions and ways of thinking. Here's an example: late in the batch, we were talking about monetization problem since the vast majority of our users were free users. We didn't want to turn cripple AWW free functionality but couldn't find a compelling reason for many of them to upgrade. Our mentor asked us point-blank: “I know you're against putting ads there, but why not give it a shot and see what happens?”

The ads

This wasn't new idea. This was pretty old idea that I shot down time and again because as a consumer I dislike ads, especially the creepy follow-you-around targeted kind, and I subscribe to the “if you're not paying for the product, you're the product” line of thought. I also thought ads would clutter useful screen estate – on a whiteboard, you want the whiteboard area to be as large as possible. So I always dismissed this idea without a second though, and even my cofounder didn't convince me otherwise.

But this question at the Startup School office hours got us thinking. Can we do it in a way that wouldn't screw up the experience? Perhaps. We didn't jump to it straight away but the idea kept simmering in the back of our minds.

Finally, in the spring of 2018, we gave it a go. The idea was simple: free users get ads. Subscribe and besides getting all the premium features, you also get rid of ads. The rationale was that a lot of people don't actually mind ads for free stuff and for those that do, well, there's an easy way to remove them. This sounds like such a simple concept but it took us a lot of time to internalize all the rationale and the consequences of this decision.

We implemented Google Adsense in early 2018 and after a few weeks of wildly erratic earnings, it settled down to a pretty low value, with a lot of traffic showing as invalid. To this day I'm not exactly sure what was the reason.

I suspect it was some combination of us testing the ads on only 10% of users to begin with, screwing up the ad placement implementation (there were initial app modal dialogs that were showing above the ads in the first few seconds), and ad rotation (our users were on the same page for extended period of time and showing the same ad for 15 minutes makes no sense).

One by one we fixed these perceived problems. It took very long time for the invalid traffic ratio to lower down (from 50% of ads shown to some 10% of total).

In the end, ads were bringing in about a fifth of our revenue, but they also increased our conversion rate (free to paying users), and haven't materially dented our growth (users leaving in frustration because of ads was my biggest fear).

Chasm

Late 2018 was a rough time for AWW. About two years of talking to customers, building, measuring, revamping the product and the UI didn't produce the hockey-stick we were aiming for. Not even the “reasonable” 4x-5x growth we expected at the least. We were growing – but very slowly.

What did we do wrong? Maybe we didn't focus narrowly enough (we still catered to EdTech, small business users, other apps embedding us and even had an on-premises enterprise offering). Maybe we weren't aggressive enough in customer acquisition (inbound sales only, no advertising). Me spending just half of my time on the startup certainly didn't help.

We had some tough decisions to make. After a lot of soul searching we decided to trim down, letting everyone go, with Zvonimir and me staying to work on it part-time. Most of our time we spent on customer relations, inbound sales and maintaining the service.

Without our noticing it, the service still grew, though. In retrospect, we did good work on a number of things back in 2017, it just took a lot more time for the effects to become visible. By the end of 2019 we approached triple the revenue numbers from 2017 (on a slower user growth).

Preparing to sell

Okay so it wasn't a rocket to the moon we hoped for, but AWW was still a nice little side business. Trouble was, both of us also had other businesses to attend to and couldn't keep up focus both on the new ventures and AWW indefinitely – something had to go. We also didn't want to shut it down, so we started thinking about selling it.

We researched and interviewed several online business brokers and decided to go with FE International, partly on the strength of patio11's recommendations.

Preparations for going to market involved us compiling detailed financials for the previous few years, as well as answering a questionnaire about the business itself, the customers, competition, reasons for selling, and so on.

After we provided this information, FEI prepared a go-to-market prospectus (a sales brochure), we okayed it and they started reaching to their network. This preparation took several months, so by the start of March 2020. we were on the market.

In two weeks, everything changed.

The world moves online

With the pandemic surging everywhere, much of the western world started shutting down or going online. Schools started adapting their curriculum for online, businesses went full work-from-home. Everyone was on Zoom, Teams, and Meet.

Every online productivity and collaboration tool imaginable started seeing their usage numbers surge. This also happened to us. Teachers, school district IT departments, businesses, started signing up in droves.

Our user numbers, customer numbers, number of shared whiteboards, revenue – basically every important metric – skyrocketed. We were up 6x (that's 600%) by end of April compared to mid-March, and the growth didn't stop there.

We were profiting from the situation, but also wanted to help. We set up a policy to give free premium access to any educational institution until the end of the school year. Besides being a decent thing to do, we figured (correctly) it would convince many schools to stay with us the next year.

The mad IT scramble

Our systems were woefully unprovisioned for the sudden increase in load. We started provisioning new servers left and right but soon found out there were bottlenecks in how the system was organized that prevented taking full advantage of horizontal scaling.

Examples were having many smaller (unsharded) MongoDB replicas in a set instead of using fewer larger-capacity servers, having database interconnect latencies that caused MongoDB queries to be dispatched unevenly across the servers.

Then there were the sloppy things that went unnoticed for years on a smaller load, such as not cleaning up Redis temporary keys quickly enough (leading to our Redis servers quickly using up any amount of memory we threw at them), using a few database queries where a single (complex) one might do the trick, or our indexes not matching the actual usage patterns.

And then there were the subtle bugs and edge cases. As the saying goes, if you have a million users, million-in-one bugs are a daily occurrence. Although AWW is designed to gracefully withstand shorter connection problems, this didn't always work flawlessly and sometimes even increased the problem by hammering the servers too much when they were already under heavy load.

Our platform providers had their own woes. With everyone buying or renting any capacity they could get their hands on, the platforms themselves become less stable. We were hosted on Digital Ocean and although they're in general great as a platform (and I have virtually all my other stuff there), we were plagued for months by strange intermittent issues where a system of ours would just lock up under moderate load.

Lots of investigation, together with the engineering team on Digital Ocean's side, yielded no specific results, except that issues were related to the “shared” type droplets (servers) we were mostly using at the time. Switching to “dedicated” droplets helped a bit, but didn't eliminate the issue. Whether it was something we did, or there was a just a capacity and noisy neighbor problem, we'll never know. In the end, we resulted to rebooting any droplets showing signs of being locked up.

Droplet rebooting

Growing the team

Everyone has their “what did you do when the pandemic hit” story. Mine was holed up in the attic-turned-home-office in front of a computer, madly trying to keep the system running, both excited and terrified. Days were a blur – I don't remember much of anything else until some time at the end of April.

When the pandemic hit, both Zvonimir and me had other work obligations. We couldn't just drop those at a moment's notice, so for weeks we put in required hours on those obligations and every other waking moment on AWW.

It still wasn't enough. Especially the support tickets (combining the questions about our service and signing up, and problem reports) were overwhelming. I remember powering through them late at night, going to sleep, waking up early morning (usually by some server alarm going off) and then seeing another few hours' worth of tickets already waiting.

It took us about a week to decide to grow the team, and then find, hire and onboard a dedicated support person. She was able to handle most of the questions from users while escalating to us stuff like bug reports, bizdev/sales questions and other more complex inquiries.

This finally gave us some breathing room and ability to focus on the business (Zvonimir) and tech (myself) matters. In the coming months we hired another support person, two developers and a part-time designer.

Growing the business

While I was busy scaling up our tech, Zvonimir spent most of his time on Zoom calls, talking to our customers and prospective customers. Whereas previously most of our users and customers were individuals, now organizations of varying sizes were knocking on our door.

We tweaked our bigger pricing plans to better match what people were asking us for. With bigger enterprises now knocking on our door, things like service-level agreements (SLAs) and standard operating procedure (SOP) documentation needed to be set up and addressed before bigger companies could roll us out to their teams.

All of this took enormous amount of time. Deals still took weeks or months from initial contact to implementation. All of it was inbound interest but there was a lot of work for every bigger deal. Had we had a dedicated smooth-running biz team, I'm sure we would have grown twice as fast or more.

No matter. Even with a small team, by end of 2020 we've grown 10x, and that includes lull summer months when most of the EdTech activity is subdued.

To sell or not to sell?

Our explosive growth started just as were about to put ourselves on the market. Almost immediately, the old numbers didn't make sense. Users, revenue, growth numbers were nothing like the ones used to calculate a (target) sale price.

In the next several weeks we did receive several offers that would've been interesting before the growth started, but didn't make sense in the new environment. We decided to put the sale on pause until the numbers became more predictable and for the next few months didn't think about it.

By mid-summer the growth stabilized – we were still growing, but at a bit more predictable pace – so we revisited the idea to sell, as we still had a (paused) contract with our brokers.

Based on our experience, selling via brokers is easier if there's a flat or consistent small growth where future can be easily extrapolated. Coming up with the price then involves tallying up income and expenses and multiplying with a factor you come up from experience and rules of thumb.

In our case, growth was triple-digit, virtually all of the expenses were investments into growth so this pricing model breaks down. We adapted the model, updated the numbers and went to market again. Zvonimir had by that time learned the nitty-gritty details of company valuation and had actually provided the final calculation himself.

Ultimately, we haven't received any credible offers for the updated price and since we weren't under any pressure to sell, we decided not to.

Miro

Earlier in 2020, Miro had raised $50M in a Series B funding round to fuel their growth. At some point their reached out to us to see if we could cooperate and by the autumn the topic turned to the possibility of them acquiring us.

In contrast to earlier potential buyers, Miro was in the same niche as we were, with a similar product and was also a technological company. We liked the team and their product and felt Miro could be a good home for AWW. Also, the price was right based on our revenue, users and growth numbers.

It took some time to negotiate the details of the deal and go through due dilligence process. We went over dozens of drafts for both LOI (Letter of Intent) and the actual purchase agreement – although we had a “gentleman's agreement” from the business perspective, there were still a lot of details to iron out.

While we obviously had our own lawyers, tax advisors and accountants (on both Croatian and US side!), we wanted to be hands-on and understand everything, so by the end of it both Zvonimir and me read cumulatively hundreds of pages of US contract legalese.

Due dilligence also got complicated: although AWW, Inc. was a US company, both Zvonimir and me were residents of Croatia and we had contractors from all over the place, not to mention the original intellectual property (IP) traced back to my development agency (Good Code) from the times it was just another project there. Although we had IP purchase paper trail, some of it was in Croatian and had to be translated, and the lawyers wanted to triple-check the rest.

Finally, the deal closed and on February 22nd, 2021. we publicly announced that AWW is joining Miro!

AWW joins Miro

Since both AWW and Miro use custom in-house built whiteboarding functionality, there wasn't much sense in continuing to develop both products, and Miro decided to wind the technology powering AWW down.

So we set out to migrate existing customers: for enterprise customers there was more work involved on our side, while smaller users received 1 year free equivalent subscription to Miro and guidance on how to export their AWW whiteboards and upload them to Miro. All users were given 5 months to do the transition, and we timed it such that education organizations were able to finish their school year on AWW.

Miro also took our experience and lessons learnt from AWW in designing and developing a free version, so although no AWW code is directly used, our focus on simplicity, immediacy and ease of use continues.

Looking back

If a time-traveler told me ten years ago how the story would unfold, I would laugh them off. What started as a hobby experiment evolved into one of the global leaders in its niche, was used by millions and changed our lives forever.

I like to think that in a tiny way we improved the lives of all our users – be it a kid doodling something in a collaborative drawing class, a roofing company making it easier for their customers to sketch what they need, or a developer being able to discuss software architecture on a confcall – we made those moments easier and more fun.

And hey, what more can you ask for.

Is software development a proper engineering discipline? On one hand, it looks like it should be – computer science is a rigorous, hard, math-based science. The hardware is deterministic.

As anyone involved with the discipline will attest, though, it's sorely lacking. Just Google for “if programmers made ...” or “if software engineers built ...” for the cheeky, but not entirely untrue, comparisons.

Why are we so bad at it? Is it because we're really struggling to be something we're not? Or are there other reasons we're failing?

Let's do a cheeky comparison of our own.

Bridging the gap

Suppose you want to build a bridge. Right off the bat, you know a few things:

  • which two places across the water you want to connect
  • how many lanes and of which type (car, train, bike, pedestrian) you want to support
  • terrain on both sides and the weather patterns
  • typical building materials for the bridge and their properties

Based on this, you can design the bridge architecture. It'll probably go through a few iterations – maybe number of lanes will change based on the cost projections.

During the design, you'll never:

  • cheap out on the architects – it's vital you get this right and the design phase cost is small compared to the overall project budget
  • rush the design phase – most of the calendar time is spent chasing around stakeholders (getting people to decide on building a bridge in the first place, chasing permits, etc) so there's no need to rush the architects

Once everyone signs off on the plans, you know exactly how and what to build, down to the bill of materials. Then you go and build the bridge.

Once it is built, it is done, except for regular maintenance. The terrain doesn't change, the materials don't change their properties, and you definitely don't attempt to move the bridge or add another lane in a year.

You also don't expect terrorists to be attacking the bridge, and you usually don't account for once-in-a-thousand-years natural disasters. If you try to, you'll be accused (correctly) of over-engineering and being insecure. There are standard safety factors and you don't need to go over them except for a very good reason.

I'm not a civil engineering myself (I have a bit of background in mechanical engineering), but I don't think any would find the above description and assumptions wrong. Simplistic, yes, wrong, probably not.

In contrast, in software development you often:

  • don't know the exact requirements, or they will change throughout the development process
  • want to keep the software as configurable and adaptable as possible (no hard limits on the number of users or amount of data, for example)
  • can't account for all the differences in the platforms your software will run in (either native desktop/mobile, or across web browsers)
  • deal with platforms or frameworks that change rapidly and in a ways that make you rethink the design of your software
  • need to interact with poorly specified and/or poorly working 3rd party components (even if they work great, network stability can always be a problem)

What's more, with software:

  • we expect to need it to change – either to evolving business realities or to keep up with the underlying platform changes
  • we expect it to be reasonably robust against malicious attacks

In contrast to civil or mechanical engineering, when we talk about “designing the solution” in software development, we talk about having a high-level architecture with roughly fleshed-out components (what they are, what they need to interact with). It's not even close to the level of detail the architecture plans give the bridge builders.

This means that the design phase in civil engineering doesn't map to solution design in software development – it actually maps to the entire programming effort, which is the bulk of the work, cost and time in any software development project.

        +------------+-----------------------------------+
BRIDGE  |   Design   |              Building             |
        +------------+------------------------+----------+
 APP    |             Programming             |   Ops    |
        +-------------------------------------+----------+

Copying bridges

Ok, you build one bridge. How hard is to build another just like it? Well, you repeat the whole process again in its entirety. Hopefully you do reap some savings due to the fact that you're now more experienced and have established supply chain connections, but that's it. The majority of work is still there and if everything else stays the same, the budget won't be very different.

In software development that's – you just make (or deploy) a new copy. Done!

What does this mean for builders themselves? This means in civil engineering you get more money for building (executing on the plan), while in software development you get more money for developing the solution (the design part). You just don't hear about multinational mega-corporations of architects on a level with FAANG.

Interestingly, the multinational mega-corporations handling ops do exist (PaaS, IaaS, app stores). They combine infrastructure, distribution and operations – in our comparison, in the civil engineering world they'd be an amalgam of builders, utilities and shipping companies all fused into one.

Cost of failure

If a bridge collapses, that's a huge problem, even if noone gets hurt. If software malfunctions, in most cases it's not a big deal (sometimes it is and then it's newsworthy). More often, if it works but badly, people scrape by.

Nowadays you can even fix the bug and push an over-the-air update – no need to recall all the cars of planes you've built with the defect.

This means the cost of failure (either catastrophic, or failure to live up to the expectations) is much lower in software than in either civil or mechanical engineering. That again means the return on investment (ROI) on any effort mitigating that cost is small, discouraging spending time and money on it.

So software today is buggy becase it can be. It's slow because it can be. It's insecure because it can be. It's not due to any lazyness of programmers, incompetence of managers or malice of big bosses. It's how the world works – if you spend too much on quality where not required, your competitor can spend the same money to grow faster with no downside.

Yet it is not obvious that it is unequivocally a bad thing. Consider the failures of famous projects with (near) unlimited budgets and perfectionism, such as Duke Nukem Forever. It makes as little sense to overengineer a software product as it would be for a building. Tech moves at breathtaking speed, and we've come so far in just a few decades, because it often can “move fast and break things”. Software can be more adaptable and quicker to the changing needs.

As software becomes a vital part of more and more things in the real world, the costs of failure rise. Whereas you might be only slightly annoyed at having to restart your Facebook app, you really don't want any trouble from the software controling your two-tonne car while you're doing 80 on a highway.

If that means more software will need to move slowly and avoid breaking things. And maybe we'll be more like the bridge builders.

Internet, in 2018, was not a safe place.

By this I don’t mean spam arriving in our inbox, viruses or malware lurking in software downloaded from less-reputable places, or phishing sites masquerading as our favorite e-commerce platforms.

These risks are real, but well understood and widely recognized. However, in the past years there has been an increasing evidence for, and occurrence of, completely different kinds of risk that most of us online are exposed to.

Examples of these are pervasive tracking of behavior online, appropriation of personal data by the apps or sites we use, data breaches, and junk media optimized to maximize engagement.

Before I go over each of these in more detail, a disclaimer: I don’t think everyone’s out to get me, or that big corporations such as Google or Facebook are inherently evil. I do think that companies, big and small, are incentivized to behave in ways that create or increase these risks. That is, the default is to behave in a way that makes things worse.

Read more...

Dispelling one particular critique of UBI

Universal Basic Income (UBI) has started appearing with increasing regularity in research and experiments all around the world (Finland, India, …Oakland?). Of course, the scheme has both benefits and drawbacks, its proponents and critics, but in the absence of experience from a large-scale long-running UBI program, it is hard to evaluate what would actually happen.

Read more...

Voice-controlled AI assistants are advanced enough to be dangerous

Useful voice recognition, combined with AI capable of parsing specific phrases and sentences, is finally here. Amazon’s Alexa, Apple’s Siri and Google’s Assistant are showing us what the future will be like.

Read more...