Now we have the opposite problem: too much information. The challenge today is curation: we want to consume a reasonable amount of information that's relevant to us.

What's “reasonable” and what's “relevant” is tough to define as it is person-specific. What's reasonable and relevant to you may not be the same as what's reasonable and relevant to me.

From curation ...

Curation of information is nothing new: newspapers, radio and TV have done it since the beginning. We relied on the media to pick what's most important and only kept the choice of a source we trusted or liked.

As the Internet grew, the amount of information grew exponentially. One of the reasons for Google's early success is that it was very good at curating this information. While the other search engines could also find 10,000 pages matching your search, Google was the best at picking what was most useful, informative and relevant.

Other services were also trying their best at curation: Netflix had a great recommendation algorithm, and have for years organized a public competition to optimize it and provide even better recommendations.

.. to filter bubbles

Over the years, as the various algorithms got improved and optimized, they got subjectively worse for us, the users. Google is in arms-race with SEO folk, Facebook prefers to serve you content that will rile you up, Netflix' algorithm seems braindead and the less we talk about YouTube recommendations the better.

Why is this? Turns out that, at some point in their lifecycle, big companies had to choose between optimizing for the user experience and optimizing for revenue. The algorithms improve allright ... but using a different metric.

Google now hyper-optimizes for what it thinks you should see (filter bubble), Facebook serves you whatever will keep you on the site for longer (mind-numbing memes interspersed with viral outrages), and Netflix brings up and center its own content that it would like you to get hooked at (and not cancel the service), not the things you might actually want to see.

Curation for the benefit of the user, has turned to serving targeted content, for the benefit of the company.

Enshittification

Enshittification is an ugly word to describe an ugly thing. Basically, it is a strategy change in an online platform where it switches from user growth (where optimizing user experience is paramount) to revenue growth (where you need to squeeze maximum income from each user).

Enshittification, or platform decay, is not limited only to Internet media companies. But it is especially visible in online startups that grew big by giving away their product or undercharging for it, in order to grow as fast and big as they can.

At some point you got to earn money, and you have to earn as much money as possible, and since you have a mostly-captive audience (there is only one Google, Facebook, YouTube or Twitter), they won't leave if the user experience is marginally worse.

And so the UX frog gets slowly cooked. People today are hooked on Facebook scrolling, Twitter mob rages, memes everywhere and 10-second dopamine hits on TikTok or YouTube Shorts. It's the sugar thing all over again.

Information detox

Faced with this, some people try limiting their consumption of this kind of content (me included, see my Junk Social and Digital hygiene posts).

This can work if you're willing to limit your information access, don't suffer from the fear of missing out (FOMO) and have the mental strength to not succumb when you're tired, bored or just open up your favorite social media app on autopilot because it's on your mobile phone home screen.

But it's hard work: you're basically building and maintaining a Great Wall between yourself and most of the modern Internet.

Curation on our terms

Can we do it better? Is there a way to again outsource curation of content that is optimized for us, the users?

Curation is hard work, whether you want to build and maintain sophisticated algorithm or AI to do it, or if you have actual people doing the work. And it's not entirely obvious how that would work without degenerating into the filter bubbles and social media we have today – that's how some of those companies started, after all.

As a technologist I do believe a solution is within a realm of possiblity. As someone who's watched Internet grow from an academia and hobbyst garden into what it is today, I am skeptical we'll reach that solution (and if you really want to be scared about the prospects of it, go read The Master Switch).

Any sufficiently advanced technology is indistinguishable from magic.

Modern AIs and ChatGPT in particular look like magic to many people. This can lead to misunderstanding about their strengths and weaknesses, and a lot of unsubstantiated hype.

Learning about a technology is the best antidote. For curious computer scientists, software engineers and anyone else who isn't afraid of digging a bit deeper, I've compiled a list of useful resources on the topic.

This is basically my reading/watching list, organized from more fundamental or beginner friendly to the latest advances in the field. It's not exhaustive, but should give you (and me) enough knowledge to continue exploring and experimenting on your own.

General overviews

If you don't have a lot of time or don't know if you want to dedicate effort in learning the ins-and-outs of modern AIs, watch these first to give you a general overview:

• The busy person's intro to LLMs (for text-generation and chatbot AIs)
• Diffusion models explained in 4 difficulty levels (for image generation AIs)

Fundamentals of neural networks

The videos here provide both teorethical and hands-on introduction to the fundamentals of neural networks.

MIT Introduction to Deep Learning

A good theoretical intro is MIT's 6.S191 class lectures, especially the Introduction to Deep Learning and Recurrent Neural Networks, Transformers and Attention.

These overview lectures briefly introduce all the major elements and algorithms involved in creating and training neural networks. I don't think they works on their own (unless you're a student there, do all the in-class excercises, etc), but it's a good place to start with.

The topics discussed here will probably make your head spin and it won't be clear at all how to apply them in real life, but this will give you the lay of the land and prepare you for practical dive-in with, for example, Andrej's “Zero to Hero”.

The example code slides use TensorFlow. Since Andrej's course uses PyTorch, going through both sets of lectures will expose you to two most popular deep learning libraries.

Neural Networks: Zero to Hero

An awesome practical intro is the Neural Networks: Zero to Hero course by Andrej Karpathy (he also did the busy person's intro to LLMs linked above).

Andrej starts out slowly, by spelling out the computation involved in forward and backward passes of the neural network, and then gradually builds up to a single neuron, a single-layer network, multi-layer perceptron, deep networks and finally transformers (like GPT).

Throughout this, he introduces and uses tools like PyTorch (library for writing neural networks), and Jupyter Notebook, and Google Collab. Importantly, he first introduces and implements a concept manually, and only later switches to a PyTorch API that provides the same thing.

The only part where things look a bit rushed is the (currently) last – GPT. There's so much ground to cover there that Andrej skips over some parts (like the Adam optimization algorithm) and quickly goes over the others (self-attention, cross-attention).

Overall a great guide. You only need to know the basics of Python, not be afraid of math (the heaviest of which is matrix multiplication which is spelled out), and do the excercises (code along the videos) without skipping the videos that don't seem exciting.

Understanding Word2vec

Both the MIT and Andrej's lectures touch on embeddings (the way to turn words into numbers that a neural net can use) only lightly. To deepen your understanding, Illustrated word2vec article explains *word2vec+, a popular word embedding algorithm, step by step. It also features a video explanation for those that prefer it to text.

Another good lecture on the topic is Understanding Word2vec.

CNNs, autoencoders and GANs

The MIT lectures mention earlier also contain lessons on Convolutional Neural Networks, autoencoders and GANs, which are important building blocks in neural networks used in vision.

Again, these are high level overviews and although formulas are present, the lectures more give an overview of the algorithms without going into too much detail. That makes them ideal prequel to the Practical Deep Learning course by Fast.ai.

Diffusion models

Diffusion models build on top of CNNs to create image-generating and manipulating AI models. Beyond the general overview linked earlier, the Introduction to Diffusion Models for Machine Learning is a deep dive into exactly how they work.

Practical Deep Learning

Practical Deep Learning is a free course by Fast.ai that has (current count) 25 lectures covering both high-level practical parts of neural networks and the underlying fundamentals.

In particular in Part 2 they cover “zero to hero” on Stable Diffusion, a powerful image-generation AI model.

Large Language Models

These resources go in-depth about constructing and using large language models (like GPT):

Transformers

Andrej's course goes over the transformer (building blocks of GPT) architecture, but the complexity makes it easy to get lost at first pass. To solidify your understanding of the topic, these two are super useful:

The Illustrated Transformer describes the transformer (building blocks of GPT) in detail while avoiding tedious math or programming details. It provides a good intuition into what's going on (and there's even an accompanying video you may want to watch as a gentler intro).

Follow that up with The Annotated Transformer which describes the scientific paper that introduced Transformers and implements it in PyTorch. Since it's 1:1 annotation of the paper, you need a lot of understanding already so only attempt going through this once you've watched both Andrej's course and once you've read and understood the Illustrated Transformer.

Reinforcement Learning through Human Feedback

Language models are good at predicting and generating text, which is different from answering questions or having a conversation. RLHF is used to fine tune the models to be able to communicate in this way.

Illustrating Reinforcement Learning through Human Feedback from folks at HuggingFace (an open source AI community) provides a good overview of RLHF. They also did a webinar based on it (video is the complete webinar, link jumps directly to start of RLHF description) based on the blog post.

If you want to dive deeper, here's the InstructGPT paper from OpenAI, which basically describes the method they used to create ChatGPT out of GPT3 (InstructGPT was a research precursor to ChatGPT).

Fine-tuning

Fine-tuning allows us to refine or specialize an already (pre)-trained LLM to be better at a specific task (RLHF is one example).

Sebastian Rashka's Finetuning Large Language Models explains a few common aproaches to finetuning, with code examples using PyTorch. He follows that up with Understanding Parameter-Efficient LLM Finetuning, a blog post discussing ways to lower the number of parameters required, and an in-depth article about Parameter-Efficient LLM Finetuning with Low-Rank Adaptation (LoRA).

Full courses

If you want a really deep dive (undergrad or higher level), follow these courses including doing the excercises / playing around with the code:

• Neural Networks Zero to Hero (Andrej Karpathy)
• Introduction to Deep Learning (MIT)
• Introduction to Deep Learning (Sebastian Raschka)
• Practical Deep Learning for Coders (Fast.ai)

This is a living document (ie. it's a work in progress and always will be). Come back in a few weeks and check if there's anything new.

You don't need to worry, and here's why.

ChatGPT is scarily good. It really is. No it's not better at web search than Google (yet), and it's nowhere near being sentient. But for the tasks where it makes sense, it's very good. So I'm not going to tell you that you don't need to worry because it's a useless tool.

ChatGPT (and other AIs) is a tool, in the same way a calculator is a tool or a compiler is a tool. The word “calculator” used to refer to people doing number crunching. My first calculator was a handheld device. Now it's just an app. Yes, calculators, the people, lost their “job” doing mind-numbing number crunching, but they were able to work on more interesting problems in math, physics, or what have you.

Same with compilers. When compilers were invented, people were furious that someone thought a machine could do a better job than an expert programmer at crafting assembly code. Today almost nobody writes directly in assembly, except in rare cases where that makes sense. But “assembler programmers” didn't lose their jobs, they just became “C programmers” or “Lisp programmers”.

It is the same with the new AI models. They are very effective on a whole other level than just number crunching or compiling software, but at the end of the day they're just tools, like programming languages, libraries or APIs.

If you've been a programmer for more than a few years, you know you always need to learn new stuff and stay on top. You'll need to invest some time to learn what ChatGPT and others can do, or can't. What you don't want to do is completely ignore the trend, or (equally bad) think it'll solve all your problems (or put you out of business). And as John Carmack said in a recent tweet keep your eyes on the delivered value and don't over focus on the specifics of the tools.

I've been a programmer for some 30 years (20 or so professionally) and my usual reaction to new stuff is “oh, so they're reinventing that particular wheel again”. Yet the current crop of AIs gets me really excited, like I was a kid again uncovering the vast potential of what you can do with a machine that you can order around! I've been playing with ChatGPT and it makes me faster and it makes programming (more) fun!

Not by writing my code – I don't use it like that because it does produce bugs and it can hallucinate stuff. I use it to explore (how I might go about doing X), and to quickly recall something I forgot (how a particular API or library function is used, for example). As Simon Willison (of Django and Datasette fame) puts it AI-enhanced development makes me more ambitious with my projects.

This is just a beginning, and we're just seeing a boom in integrating these AIs into everything else. Copilot, Bing search, are the big names but people are experimenting with integrations with anything under the sun (I made a service that creates API backends based on your project description, for example). Time will tell which of these will be truly useful, but I have no doubt there will be a lot of them.

AI is a tool, with limitations, but with a lot of potential. It would be a shame not to use it effectively. It won't put you out of a job, it will make your job better.

Examples include “Building a Cache in Elixir”, “Helix: Neovim inspired editor, written in Rust” or “Text formatting in C++ using Libc++”.

Since these “in $LANGUAGE” articles come up fairly often, I was curious to see what we can glimpse about programming language popularity just by counting them. This is easy to do using HN Search API provided by Algolia.

Methodology

I used it to search for stories in the past year, where the phrase “in $LANGUAGE” appears in the story title. The search included C, C++, C#, Clojure, Dart, Elixir, Erlang, F#, Go, Haskell, Java, JavaScript, Kotlin, Lisp, Lua, PHP, Python, Ruby, Rust, Scala, Scheme, Swift, TypeScript, and Zig languages.

I also included Racket Scheme and added its numbers to Scheme. I'd have included Common Lisp as well, but there were no stories mentioning it explicitly in the title in the past year. Also, since C would match both C++ and C# results in this simple string search, I deduplicated the results to get the correct count.

I ranked the languages in three ways: by the number of stories, number of comments, and sum of points. The results were fairly consistent across all three metrics.

If you're interested in running this yourself, this is the script I used (you'll need the requests package to run it). It downloads and caches search results as JSON in /tmp/hn, sorts the languages, and prints the results.

I ran the script on October 12th, 2022.

And the winner is ...

... Rust, by a wide margin. Here are the full results:

By the number of stories:

1. Rust (573 stories)
2. Python (375 stories)
3. Go (332 stories)
4. JavaScript (200 stories)
5. C (143 stories)
6. C++ (136 stories)
7. TypeScript (79 stories)
8. Java (76 stories)
9. Ruby (57 stories)
10. Elixir (49 stories)
11. Haskell (48 stories)
12. Swift (41 stories)
13. C# (32 stories)
14. PHP (31 stories)
15. Zig (27 stories)
16. Clojure (23 stories)
17. Scheme (13 stories)
18. F# (12 stories)
19. Kotlin (9 stories)
20. Erlang (8 stories)
21. Lisp (8 stories)
22. Lua (8 stories)
23. Dart (4 stories)
24. Scala (4 stories)

By the number of comments:

1. Rust (6384 comments)
2. Go (3394 comments)
3. C (3027 comments)
4. Python (2783 comments)
5. JavaScript (1583 comments)
6. C++ (771 comments)
7. Zig (735 comments)
8. Java (615 comments)
9. Ruby (396 comments)
10. TypeScript (341 comments)
11. Haskell (301 comments)
12. PHP (223 comments)
13. Scheme (177 comments)
14. Swift (165 comments)
15. C# (151 comments)
16. Elixir (133 comments)
17. Clojure (82 comments)
18. Lua (75 comments)
19. Lisp (53 comments)
20. F# (45 comments)
21. Erlang (39 comments)
22. Kotlin (7 comments)
23. Dart (1 comments)
24. Scala (0 comments)

By points:

1. Rust (16887 points)
2. Go (7144 points)
3. Python (7048 points)
4. C (5422 points)
5. JavaScript (3644 points)
6. Zig (2427 points)
7. C++ (1846 points)
8. Java (1595 points)
9. Ruby (1044 points)
10. TypeScript (998 points)
11. Haskell (846 points)
12. Elixir (751 points)
13. Scheme (549 points)
14. Swift (447 points)
15. PHP (402 points)
16. Clojure (365 points)
17. C# (334 points)
18. F# (331 points)
19. Erlang (322 points)
20. Lua (238 points)
21. Lisp (154 points)
22. Kotlin (34 points)
23. Dart (14 points)
24. Scala (7 points)

Hacker News loves Rust

Go, Python and JavaScript are quite popular, but Rust absolutely blows the competition out of the water. It had almost as much stories and comments as the languages in second and third place combined, and was awarded even more points.

If you're a regular HN reader, this isn't surprising. As a modern language with passionate community, Rust is invariably mentioned in most conversations about languages in general, and stories about using Rust for a new project or in a new are always exciting.

And I'm not even counting the “Rust in the Linux kernel” theme, which alone got around 600 points across various stories in the past year.

The new kid on the block is ... Zig

What did surprise me is the relative popularity of Zig, a fairly young language with still a relatively small comunities – I'll be sure to keep an eye on it.

—

(comment on Hacker News) | (follow me on Twitter)

The pandemic was a global tragedy and a big disruptor to every aspect of the lives of people everywhere. One of the few silver linings was that it forced a rethinking of the importance of location for the knowledge worker and what “office” actually means.

In my two decades of professional experience, I spent about half of it working remotely. The other half we had a small office (10 to 15 people), but mostly worked with clients overseas. In my time as both an individual contributor, and manager of dev teams, I had to handle the challenges arising from, and enjoy the advantages of, remote work.

My view is naturally colored by my experience. I am a software engineer and have worked in IT my entire career. While I believe most of what I say here can be generalized, there are nuances for every profession.

Location, location, location

Let's get obvious out of the way. Not all work is – or will be – possible to do remotely.

Although technology will continue to push the limits (remote truck driving, anyone?), many jobs, from surgeons to shopkeepers to street sweepers, still require people to be in a certain location to do their job. This does not mean those jobs are lesser, or that they will lose out in the “remote work revolution” – quite the opposite. Fewer people commuting to work and crowding in hot locations means less traffic, less stress and more room for those that choose to – or have to – be there.

Bare necessities

My job as a software developer consists mostly of reading from a computer screen and typing on a keyboard, with the occasional use of a mouse. I can do that anywhere on the planet, provided there is electricity and an adequate internet connection. In reality, what I consume (as a work input) and produce (work output) is information, and that's easily sent across the world.

The same holds mostly true for most knowledge workers. When push came to shove (the pandemic lockdowns), a lot of people realized the work they do can be performed remotely.

Hierarchy of needs

Of course, that's not all there is. We still need information, often implicit, from our coworkers, about the problem we're solving, and the wider context our company, customers, and clients are operating on. Much of this is recorded poorly, or not at all. Remote interactions are harder than face-to-face and Zoom fatigue is real. And sometimes we need specialized office or lab equipment.

And as much as we love working in our pajamas, we might miss office chit-chat or getting that energized feeling from being a member of a well-oiled team of awesome people doing great work together.

Back to life, back to reality, back to the here and now

Now that things are normalizing a bit (I know it's a stretch to use that word in the middle of 2022), companies are starting to mandate back-to-office or hybrid work. Conspiracy theories about reasons are many, from wanting to micromanage workers better, to making sure expensive office leases are utilized, to being a sneaky way to lay people off without actually giving them the pink slip.

However this is resolved, the genie is out of the bottle. Remote work has previously been regarded as a unique perk or very specific work arrangement. Now, wherever it's allowed or not in a particular company, it's normal. I think that's a good thing.

Checking our privilege

Us IT folks have it easy. The job market is hot and there are plenty of opportunities for good engineers. Yes, there are hiring freezes and some companies are laying off people. In most of the cases I heard of, people were quickly snatched by the competitors.

This puts us in a position to demand the ability to work remotely, and if needed, to quit and join a company that will allow that. Not everyone is so lucky and there are legions of knowledge workers who have returned (or will have to return) to the office just because the bosses decreed it.

In IT, and especially in software development, the shift is real, and companies need to adapt.

In-office vs remote work

There is a fundamental divide between in-office and remote work, and that is how the information within the company flows. Companies where in-office work is the norm can get away with much larger implicit context and rely on employees communicating directly ad-hoc, as needed.

To take full advantage, such companies maximize the overlap between the employees' working hours (9 to 5 everyone!), and try to cluster employees together (if not all the company can be in the same location, at least the teams working together) and design office space to optimize for interaction (leading to unfortunate things like open offices).

Remote-first companies focus on async work (work on your own time, communicate asynchronously with coworkers) making it possible to hire globally. Instead of spending on office space, they give workers budgets to improve their workspace or equipment. But most of all, remote-first companies take communication at all levels seriously. From office banter to all-hands meetings, stuff is happening online and is often saved or recorded for the benefit of people who are not online at that very moment.

It's easy to contrast these two extremes, but many companies will probably lie somewhere in between.

Old-style hybrid work

“Hybrid” is a word that describes two different work arrangements, depending on where the company started from.

For traditional office-based companies that are introducing hybrid work, this usually amounts to a perk allowing employees to work from home a few days per week. Often, at least two days are fixed for meetings and any other work that must be performed on-site. This type of hybrid office can easily slip back into the “office is king” mentality, where the important things still happen in the office, not everything is shared with remote workers (not necessarily due to any malicious intent), and the more you're present, the faster you'll career trajectory will be.

This was mostly the status quo for remote workers before the pandemic. Except in rare enlightened companies, remote workers were de-facto second-class citizens. The more a company now stresses that being in-office is important, the more likely it'll fall back on this pre-pandemic default.

I believe that companies that are now requiring employees to be present at least three days a week will most likely fall back to this mentality.

If they combine this with hot-desking, where employees must find themselves a (possibly different) desk each day, it will only make things worse. I fear this may be a real possibility for many, to try to have the workers back but still save on office space.

Remote-first hybrid work

Another type of hybrid-location company is remote-first which recognizes there is benefit in getting people together or making it easy for people who want to work from the office to do so. These companies understand office is just another location. Hopefully, the environment is optimized for work (quiet environment, good equipment, etc) but in terms of work communication, it holds no special place.

A company I currently work with has such an approach. They're NYC-based, but have employees throughout the US, as well as people in Latin America and Europe. The main communication channels are all digital – Slack, Zoom, Notion, and email. Scheduling Zoom meetings across time zones is a bit challenging, but they manage and are looking to improve as they grow.

From remote-as-perk to remote-first hybrid

There's a tremendous opportunity for companies that are willing to switch from the office-based (or remote-as-perk a few days per week) model to the remote-first hybrid model. They can still make use of their existing office space (possibly downsized to reap financial benefits), reconfiguring it so it is support for people who want to work from the office, or who need to come to the office.

Essentially, they can turn their offices into a sort of internal coworking space. Employees that don't want or can't work from home can still come to the office. Coworkers or entire teams can independently or ad-hoc agree to come to the office for a day or a week for brainstorming sessions, project kick-offs, or just to socialize from time to time.

Remote-first, but not remote-always

In a few places I worked as a remote developer, companies had policies to bring everyone on the team together for a few days. This was sometimes combined with everyone going to a conference but hanging out in the off-times, or for everyone to be in the same place to start a new initiative. This was always a fun experience (even when paired with hard work) and I felt energized and ready to take on new challenges.

I believe fully remote companies should try to bring their people together at least once or twice per year. This can be company-wide (for smaller organizations) or per-team (easier to do with larger companies). If the company is fully remote and doesn't have an HQ, doesn't matter: pick a nice location and if there's a conference or another interesting event, that's even better.

Remote vs overseas

A lot of companies these days say they hire remotely but within a country. For example, a job listing may say “remote, US-only”.

This is understandable: it's legally easier to hire within the country you're incorporated in, the cultural differences are smaller, and time zone difference is more manageable. it can also be harder to find remote talent since you don't know where or how to look. The stigma of hiring overseas developers as “outsourcing to cheap labor” doesn't help.

Cultural differences are real and should not be ignored. Being cognizant of the different ways people communicate (or avoid difficult subjects) is important. I've found that (respectfully) overcommunicating helps, as it reduces the potential for confusion. Time zone differences can have a larger or smaller impact, depending on whether your team's communication is more synchronous or async. If you have daily Zoom meetings and your team is spread over Australia, Europe, and the US, you'll have a lot of unhappy people. Somewhat localizing the teams, and making most communication asynchronous help.

Remote doesn't mean cheap labor

If you pay peanuts, you'll get (code) monkeys. Good developers anywhere know what they're worth. That said, the differences in cost of living are huge, and people do take into account the (in)convenience of living anywhere. $100K means a very different lifestyle to someone in San Francisco than to someone in Vienna or Manila.

Developers are ambivalent toward location-based salaries – on the one hand, we'd prefer to be paid what we're worth (based on the value we provide), not based on where we live. On the other hand, we don't accept that there are talented and skilled people that can provide the same value for a slightly (or more than slightly) lower price, in lower-income countries.

If companies were completely oblivious to someone's location, it would be natural for them to want to pay a lower price for the same value, so they'd hire in lower-income locations (within a country or overseas). But they're not. They do prefer local (or nearby) employees, and they are willing to pay a higher price for that. A direct consequence of this is that at least some portion of the salary is going to be location-based.

As knowledge workers, we should get to terms with that fact. We are paid based on the value we provide, but not in isolation, and the job market will always have an impact.

The future is already here – it's just not evenly distributed

The pandemic turned remote work from a fringe benefit to a normal, accepted, way to do your job, in cases, companies and jobs where that makes sense. The technology will keep pushing the boundaries where that's possible, and the knowledge workers will keep pushing to have that option.

Companies will have to adapt: by doubling down on office (and paying extra for it), embracing the new opportunity to broaden the talent pool, or finding a hybrid sweet spot that makes sense for both them and their employees.

There's no going back.

Throughout, millions of users have collectively spent a millennia drawing, learning, collaborating, and just having fun with AWW. It's been used in schools, FAANGs and everywhere in between. Early in 2021 we joined forces with Miro, the world's leading online whiteboard platform.

The start of the story is much humbler. I never intended or even imagined it would be a success it has been.

Origins

Ten years ago, back in 2011, netbooks were all the craze. What started some six years earlier with EeePC turned into an entire product category with Cambrian explosion of options. I got my hands on one with a curious feature – a touchscreen. At the time, system UI wasn't at all suited for clumsy fingers instead of precision controlled mouse pointer.

So there I was, back in 2011, trying to find a simple, nice, drawing program for Linux that would work well, UI-wise, with touchscreens. I didn't find any so decided to write my own. I took this as an opportunity to learn HTML <canvas> and write it as a web app.

It was all pretty rudimentary. You had a pencil in a few colors, eraser, and ability to clean the entire canvas. This first version even lacked the ability to save the result – you could always screenshot the page if you wanted to save.

This wasn't even a side project at the time – just a fun experiment. For kicks, I decided to put it up online and, after thinking for a whole two seconds, named it A Web Whiteboard. Since aww.com was taken, I used awwapp.com for the domain (nameapp.com was a popular way of getting around the domain name squatters at the time).

Sharing

A few months in, someone mentioned it'd be cool if you could actually have multiple people drawing at the same time. I took the suggestion an opportunity to again dabble in new tech. At the time, websockets were just being entering mainstream use and Node was starting its meteoric rise, so I selected it for the synchronization backend and now all-but-forgotten RPC-over-sockets library called nowjs. Being primarily a Python developer, I kept to at least something familiar, using Flask for saving the board contents.

This is how it looked like:

I spread the word around, got some mention in the press and received positive feedback, including on Hacker News and Reddit. There weren't many users back then (the whole thing comfortably fit in RAM of a rather small VPS), but each time users gathered to draw together, or shared a saved drawing on social media, more people heard about AWW.

After the launch, I maintained AWW but treated it as a side hobby, and for a couple of years most of my focus was on other things.

Embedding

One thing I did from the start is organize AWW as whiteboard widget that could be embedded into other sites, with awwapp.com being just one of the sites that embedded it.

When I started getting requests to let people add it to their sites, it was easy to do. I set up a PayPal subscribe button and wrote up a simple doc on how to embed the widget. When someone subscribed, I'd add them to the accounts “database” (actually a JSON file) and gave them an unique API key to embed with.

To add whiteboard to their sites, my customers would include jQuery and add the following snippet:

<div id="wrapper" style="width: 800px; height: 600px;"></div>
<script src="http://static.awwapp.com/plugin/1.0/aww.min.js"></script>
<script>
    $('#wrapper').awwCanvas({apiKey: YOUR_API_KEY});
</script>

After a time, this netted me a few hundred bucks a month, which was nice but nothing I'd base my business on – or so I thought.

I also started getting more and more serious questions about using the whiteboard on awwapp.com for business, tutoring, etc, as well as feature requests. Most of my customers were tutors using it together with Skype to provide online lessons to their students. They often didn't have a site to embed the whiteboard on, were happy to use awwapp.com version and willing to pay for it.

The rewrite

As I started treating AWW as a viable business in its own right, I started thinking about rewriting it.

The old JSON file approach was getting unwieldy, as was the fact that all active board were being kept in memory, limiting growth. Finally, the old nowjs framework was unmaintained meaning we were stuck on an extremely old Node version (0.6 if I remember correctly).

I decided on a full rewrite with redesigned architecture and using modern (for the time!) JavaScript features – stuff like promises, the socket.io library and using code bundler tools. For database we selected MongoDB, primarily for ease of use and ease of clustering/failover. We also switched from a single server (and thus single-point-of-failure) to a load-balanced system with full failover (using haproxy for web server load balancing).

This was an explosion in ops complexity. Mostly warranted, but I definitely over-engineered some things, and hit many of the novice mistakes when scaling out.

We started the rewrite in the summer of 2014 and deployed the new version in spring of 2015. I say “we” because I had help – this was now a proper project internal to my dev agency with more people involved. However it still wasn't a priority and was done at off-times and between client projects.

With the new version, we were finally able to sign up users directly on awwapp.com, again using PayPal subscriptions. We followed the freemium model, in which most of the functionality was free, but you could pay for some additionally useful features.

Ease of use

Throughout entire lifetime of AWW, I had one major focus, sometimes to the detriment of other aspects of the app: it had to be easy to use. Really easy to use. So easy a child could use it.

We did that successfully – in fact, many children did use it in kindergarten and preschool. It was great seeing pictures teachers shared of kids doing a collaborative class, or exploring maths using our whiteboard. “This is why I'm doing this”, I'd say.

For example, to start drawing on AWW you only had to click one button, labelled Start drawing, and less than a second later, you're drawing. And the only reason there was some text and a button to click first, was so people coming first time to AWW knew what the site was about! Before this, people coming up on an empty board were sometimes confusing what that was for.

Word of mouth

One thing we didn't do (except in a small experiment at the start with StumbleUpon) until much later, is promote or advertise AWW. In fact, I did hardly any outreach at all.

Virtually all of the (slow and steady) growth was entirely through word of mouth – our users telling others about us. This was helped by the fact that AWW is a collaboration platform, so users naturally wanted to share the drawing activity or results with others.

Combined with the ease of use and the fact most of the features were free, this made AWW a favorite whiteboard for many teachers and tutors. They were telling each other about it, featuring it in teacher seminars and doing video tutorials on it! This showed me how important and useful AWW was for a growing number of people and really warmed my heart.

Spinning off into a separate startup

Usage slowly but steadily grew over the years, to the point that there was simply too much to do on maintenance, customer support, inbound sales and further development, to treat AWW as a side-project.

I also realized that, as a technical person, I should get new people on board that will have “skin in the game”, a fresh perspective and complementary set of skills such as business development, sales.

I met Zvonimir at local startup event where he was just describing the challenges of his previous startup attempt. Zvonimir had tech chops and biz acumen in equal amount, combined with startup experience, but what I most liked was the levelheadedness when thinking about the realities of building (and selling!) a product in an uncertain environment.

We hit it off immediately and after some time I managed to convince him to join me as a cofounder. I consider this to be the best decision (and best sale!) I made throughout the development on AWW.

We spun off AWW as a separate startup in late 2016, incorporated it in the US using Stripe Atlas, brought several more people on board (a developer, a designer, and a marketing/support person) and set to work. We had three priorities:

The plan

First, to figure out who our users were, why they used us and what we can do to make us more useful to them. This involved identifying different markets we were playing in, such as EdTech, enterprise collaboration, etc. Up until that point we “lucked out” in that our product was useful – now we set to deliberately discover our customers and optimize our approach (feature set and messaging) towards them.

Secondly, to improve our user experience. AWW was simple and easy to use, but not so simple to figure out all the capabilities it had, or indeed why you'd want to pay for it. The more advanced “premium” functionality was hidden away unless you were a premium user, so many of our free users didn't even realize there was more to it.

Also, as our features and supported use cases grew (and were expected to grow much further), bolting them onto the old interface made it more and more clunky over time. We set out to rethink the UI from scratch, still respecting our old imperatives (simplicity) but incorporating the lessons learned.

Third priority was to start working on our marketing, sales and business development, and to improve our customer support.

Up until then, customer support meant me answering email questions with a few days' delay. We aimed to lower the reply time to within a business day and to be active on more channels, such as Twitter and Facebook.

The online presence tied in to marketing. We didn't do any advertising, aside from a few experiments to test the waters and measure potential interest for various whiteboarding use cases. However, we did want to be present on Twitter, Facebook, LinkedIn, reaching out to blogs reviewing us, talk with teachers recommending us in workshops, and this tied in nicely with the customer relations part of the plan.

The funding

Now we were in business. We had several people and everyone worked exclusively on AWW, except me. I spent half my time on AWW and half managing my software development agency GoodCode.

Even though Croatian salaries are nowhere near US (and especially Silicon Valley) ones, having more people still meant more expenses. What was previously a profitable side project now turned into a startup that wasn't yet ramen profitable. I was funding the startup from the profits from my agency so we didn't need to worry about runway for the time being, but were still open to investments.

Our rough plan was to ramp up the product and get to profitability within twelwe months. Of course we hoped for much more – our valiant efforts will result in a hockey-stick growth and back to black in three months and then sky's the limit!

That, of course, didn't happen.

YC School

At that time we applied to several accelerators, including YC, but didn't get in. Though one of the reasons was for funding, our primary reason was the advice, learning experience, networking and fresh ideas.

We also applied to the first YC Startup School in 2017 and did get in! The online MOOC-style course combining online video lectures, weekly check-ins, discussions with mentors and our fellow founders were phenomenal. I imagine the intensity is nowhere near a “full” YC experience, but it was tremendously helpful for us.

First, the weekly check-ins did wonders for our accountability. We had metrics before and we did look at what we've achieved (or not) previously, but not in as critical and clear way we had to for Startup School. We soon found out our old non-actionable vanity metrics were pretty much worthless and that we didn't even have an easy way to get to the right numbers. Focusing on the real was like wiping your glasses (or the window) and actually seeing what was going on.

Weekly discussions were as valuable. We received (and gave out) thoughtful and probing questions which made us reflect on our implicit assumptions and ways of thinking. Here's an example: late in the batch, we were talking about monetization problem since the vast majority of our users were free users. We didn't want to turn cripple AWW free functionality but couldn't find a compelling reason for many of them to upgrade. Our mentor asked us point-blank: “I know you're against putting ads there, but why not give it a shot and see what happens?”

The ads

This wasn't new idea. This was pretty old idea that I shot down time and again because as a consumer I dislike ads, especially the creepy follow-you-around targeted kind, and I subscribe to the “if you're not paying for the product, you're the product” line of thought. I also thought ads would clutter useful screen estate – on a whiteboard, you want the whiteboard area to be as large as possible. So I always dismissed this idea without a second though, and even my cofounder didn't convince me otherwise.

But this question at the Startup School office hours got us thinking. Can we do it in a way that wouldn't screw up the experience? Perhaps. We didn't jump to it straight away but the idea kept simmering in the back of our minds.

Finally, in the spring of 2018, we gave it a go. The idea was simple: free users get ads. Subscribe and besides getting all the premium features, you also get rid of ads. The rationale was that a lot of people don't actually mind ads for free stuff and for those that do, well, there's an easy way to remove them. This sounds like such a simple concept but it took us a lot of time to internalize all the rationale and the consequences of this decision.

We implemented Google Adsense in early 2018 and after a few weeks of wildly erratic earnings, it settled down to a pretty low value, with a lot of traffic showing as invalid. To this day I'm not exactly sure what was the reason.

I suspect it was some combination of us testing the ads on only 10% of users to begin with, screwing up the ad placement implementation (there were initial app modal dialogs that were showing above the ads in the first few seconds), and ad rotation (our users were on the same page for extended period of time and showing the same ad for 15 minutes makes no sense).

One by one we fixed these perceived problems. It took very long time for the invalid traffic ratio to lower down (from 50% of ads shown to some 10% of total).

In the end, ads were bringing in about a fifth of our revenue, but they also increased our conversion rate (free to paying users), and haven't materially dented our growth (users leaving in frustration because of ads was my biggest fear).

Chasm

Late 2018 was a rough time for AWW. About two years of talking to customers, building, measuring, revamping the product and the UI didn't produce the hockey-stick we were aiming for. Not even the “reasonable” 4x-5x growth we expected at the least. We were growing – but very slowly.

What did we do wrong? Maybe we didn't focus narrowly enough (we still catered to EdTech, small business users, other apps embedding us and even had an on-premises enterprise offering). Maybe we weren't aggressive enough in customer acquisition (inbound sales only, no advertising). Me spending just half of my time on the startup certainly didn't help.

We had some tough decisions to make. After a lot of soul searching we decided to trim down, letting everyone go, with Zvonimir and me staying to work on it part-time. Most of our time we spent on customer relations, inbound sales and maintaining the service.

Without our noticing it, the service still grew, though. In retrospect, we did good work on a number of things back in 2017, it just took a lot more time for the effects to become visible. By the end of 2019 we approached triple the revenue numbers from 2017 (on a slower user growth).

Preparing to sell

Okay so it wasn't a rocket to the moon we hoped for, but AWW was still a nice little side business. Trouble was, both of us also had other businesses to attend to and couldn't keep up focus both on the new ventures and AWW indefinitely – something had to go. We also didn't want to shut it down, so we started thinking about selling it.

We researched and interviewed several online business brokers and decided to go with FE International, partly on the strength of patio11's recommendations.

Preparations for going to market involved us compiling detailed financials for the previous few years, as well as answering a questionnaire about the business itself, the customers, competition, reasons for selling, and so on.

After we provided this information, FEI prepared a go-to-market prospectus (a sales brochure), we okayed it and they started reaching to their network. This preparation took several months, so by the start of March 2020. we were on the market.

In two weeks, everything changed.

The world moves online

With the pandemic surging everywhere, much of the western world started shutting down or going online. Schools started adapting their curriculum for online, businesses went full work-from-home. Everyone was on Zoom, Teams, and Meet.

Every online productivity and collaboration tool imaginable started seeing their usage numbers surge. This also happened to us. Teachers, school district IT departments, businesses, started signing up in droves.

Our user numbers, customer numbers, number of shared whiteboards, revenue – basically every important metric – skyrocketed. We were up 6x (that's 600%) by end of April compared to mid-March, and the growth didn't stop there.

We were profiting from the situation, but also wanted to help. We set up a policy to give free premium access to any educational institution until the end of the school year. Besides being a decent thing to do, we figured (correctly) it would convince many schools to stay with us the next year.

The mad IT scramble

Our systems were woefully unprovisioned for the sudden increase in load. We started provisioning new servers left and right but soon found out there were bottlenecks in how the system was organized that prevented taking full advantage of horizontal scaling.

Examples were having many smaller (unsharded) MongoDB replicas in a set instead of using fewer larger-capacity servers, having database interconnect latencies that caused MongoDB queries to be dispatched unevenly across the servers.

Then there were the sloppy things that went unnoticed for years on a smaller load, such as not cleaning up Redis temporary keys quickly enough (leading to our Redis servers quickly using up any amount of memory we threw at them), using a few database queries where a single (complex) one might do the trick, or our indexes not matching the actual usage patterns.

And then there were the subtle bugs and edge cases. As the saying goes, if you have a million users, million-in-one bugs are a daily occurrence. Although AWW is designed to gracefully withstand shorter connection problems, this didn't always work flawlessly and sometimes even increased the problem by hammering the servers too much when they were already under heavy load.

Our platform providers had their own woes. With everyone buying or renting any capacity they could get their hands on, the platforms themselves become less stable. We were hosted on Digital Ocean and although they're in general great as a platform (and I have virtually all my other stuff there), we were plagued for months by strange intermittent issues where a system of ours would just lock up under moderate load.

Lots of investigation, together with the engineering team on Digital Ocean's side, yielded no specific results, except that issues were related to the “shared” type droplets (servers) we were mostly using at the time. Switching to “dedicated” droplets helped a bit, but didn't eliminate the issue. Whether it was something we did, or there was a just a capacity and noisy neighbor problem, we'll never know. In the end, we resulted to rebooting any droplets showing signs of being locked up.

Growing the team

Everyone has their “what did you do when the pandemic hit” story. Mine was holed up in the attic-turned-home-office in front of a computer, madly trying to keep the system running, both excited and terrified. Days were a blur – I don't remember much of anything else until some time at the end of April.

When the pandemic hit, both Zvonimir and me had other work obligations. We couldn't just drop those at a moment's notice, so for weeks we put in required hours on those obligations and every other waking moment on AWW.

It still wasn't enough. Especially the support tickets (combining the questions about our service and signing up, and problem reports) were overwhelming. I remember powering through them late at night, going to sleep, waking up early morning (usually by some server alarm going off) and then seeing another few hours' worth of tickets already waiting.

It took us about a week to decide to grow the team, and then find, hire and onboard a dedicated support person. She was able to handle most of the questions from users while escalating to us stuff like bug reports, bizdev/sales questions and other more complex inquiries.

This finally gave us some breathing room and ability to focus on the business (Zvonimir) and tech (myself) matters. In the coming months we hired another support person, two developers and a part-time designer.

Growing the business

While I was busy scaling up our tech, Zvonimir spent most of his time on Zoom calls, talking to our customers and prospective customers. Whereas previously most of our users and customers were individuals, now organizations of varying sizes were knocking on our door.

We tweaked our bigger pricing plans to better match what people were asking us for. With bigger enterprises now knocking on our door, things like service-level agreements (SLAs) and standard operating procedure (SOP) documentation needed to be set up and addressed before bigger companies could roll us out to their teams.

All of this took enormous amount of time. Deals still took weeks or months from initial contact to implementation. All of it was inbound interest but there was a lot of work for every bigger deal. Had we had a dedicated smooth-running biz team, I'm sure we would have grown twice as fast or more.

No matter. Even with a small team, by end of 2020 we've grown 10x, and that includes lull summer months when most of the EdTech activity is subdued.

To sell or not to sell?

Our explosive growth started just as were about to put ourselves on the market. Almost immediately, the old numbers didn't make sense. Users, revenue, growth numbers were nothing like the ones used to calculate a (target) sale price.

In the next several weeks we did receive several offers that would've been interesting before the growth started, but didn't make sense in the new environment. We decided to put the sale on pause until the numbers became more predictable and for the next few months didn't think about it.

By mid-summer the growth stabilized – we were still growing, but at a bit more predictable pace – so we revisited the idea to sell, as we still had a (paused) contract with our brokers.

Based on our experience, selling via brokers is easier if there's a flat or consistent small growth where future can be easily extrapolated. Coming up with the price then involves tallying up income and expenses and multiplying with a factor you come up from experience and rules of thumb.

In our case, growth was triple-digit, virtually all of the expenses were investments into growth so this pricing model breaks down. We adapted the model, updated the numbers and went to market again. Zvonimir had by that time learned the nitty-gritty details of company valuation and had actually provided the final calculation himself.

Ultimately, we haven't received any credible offers for the updated price and since we weren't under any pressure to sell, we decided not to.

Miro

Earlier in 2020, Miro had raised $50M in a Series B funding round to fuel their growth. At some point their reached out to us to see if we could cooperate and by the autumn the topic turned to the possibility of them acquiring us.

In contrast to earlier potential buyers, Miro was in the same niche as we were, with a similar product and was also a technological company. We liked the team and their product and felt Miro could be a good home for AWW. Also, the price was right based on our revenue, users and growth numbers.

It took some time to negotiate the details of the deal and go through due dilligence process. We went over dozens of drafts for both LOI (Letter of Intent) and the actual purchase agreement – although we had a “gentleman's agreement” from the business perspective, there were still a lot of details to iron out.

While we obviously had our own lawyers, tax advisors and accountants (on both Croatian and US side!), we wanted to be hands-on and understand everything, so by the end of it both Zvonimir and me read cumulatively hundreds of pages of US contract legalese.

Due dilligence also got complicated: although AWW, Inc. was a US company, both Zvonimir and me were residents of Croatia and we had contractors from all over the place, not to mention the original intellectual property (IP) traced back to my development agency (Good Code) from the times it was just another project there. Although we had IP purchase paper trail, some of it was in Croatian and had to be translated, and the lawyers wanted to triple-check the rest.

Finally, the deal closed and on February 22nd, 2021. we publicly announced that AWW is joining Miro!

Since both AWW and Miro use custom in-house built whiteboarding functionality, there wasn't much sense in continuing to develop both products, and Miro decided to wind the technology powering AWW down.

So we set out to migrate existing customers: for enterprise customers there was more work involved on our side, while smaller users received 1 year free equivalent subscription to Miro and guidance on how to export their AWW whiteboards and upload them to Miro. All users were given 5 months to do the transition, and we timed it such that education organizations were able to finish their school year on AWW.

Miro also took our experience and lessons learnt from AWW in designing and developing a free version, so although no AWW code is directly used, our focus on simplicity, immediacy and ease of use continues.

Looking back

If a time-traveler told me ten years ago how the story would unfold, I would laugh them off. What started as a hobby experiment evolved into one of the global leaders in its niche, was used by millions and changed our lives forever.

I like to think that in a tiny way we improved the lives of all our users – be it a kid doodling something in a collaborative drawing class, a roofing company making it easier for their customers to sketch what they need, or a developer being able to discuss software architecture on a confcall – we made those moments easier and more fun.

And hey, what more can you ask for.

As anyone involved with the discipline will attest, though, it's sorely lacking. Just Google for “if programmers made ...” or “if software engineers built ...” for the cheeky, but not entirely untrue, comparisons.

Why are we so bad at it? Is it because we're really struggling to be something we're not? Or are there other reasons we're failing?

Let's do a cheeky comparison of our own.

Bridging the gap

Suppose you want to build a bridge. Right off the bat, you know a few things:

• which two places across the water you want to connect
• how many lanes and of which type (car, train, bike, pedestrian) you want to support
• terrain on both sides and the weather patterns
• typical building materials for the bridge and their properties

Based on this, you can design the bridge architecture. It'll probably go through a few iterations – maybe number of lanes will change based on the cost projections.

During the design, you'll never:

• cheap out on the architects – it's vital you get this right and the design phase cost is small compared to the overall project budget
• rush the design phase – most of the calendar time is spent chasing around stakeholders (getting people to decide on building a bridge in the first place, chasing permits, etc) so there's no need to rush the architects

Once everyone signs off on the plans, you know exactly how and what to build, down to the bill of materials. Then you go and build the bridge.

Once it is built, it is done, except for regular maintenance. The terrain doesn't change, the materials don't change their properties, and you definitely don't attempt to move the bridge or add another lane in a year.

You also don't expect terrorists to be attacking the bridge, and you usually don't account for once-in-a-thousand-years natural disasters. If you try to, you'll be accused (correctly) of over-engineering and being insecure. There are standard safety factors and you don't need to go over them except for a very good reason.

I'm not a civil engineering myself (I have a bit of background in mechanical engineering), but I don't think any would find the above description and assumptions wrong. Simplistic, yes, wrong, probably not.

In contrast, in software development you often:

• don't know the exact requirements, or they will change throughout the development process
• want to keep the software as configurable and adaptable as possible (no hard limits on the number of users or amount of data, for example)
• can't account for all the differences in the platforms your software will run in (either native desktop/mobile, or across web browsers)
• deal with platforms or frameworks that change rapidly and in a ways that make you rethink the design of your software
• need to interact with poorly specified and/or poorly working 3rd party components (even if they work great, network stability can always be a problem)

What's more, with software:

• we expect to need it to change – either to evolving business realities or to keep up with the underlying platform changes
• we expect it to be reasonably robust against malicious attacks

In contrast to civil or mechanical engineering, when we talk about “designing the solution” in software development, we talk about having a high-level architecture with roughly fleshed-out components (what they are, what they need to interact with). It's not even close to the level of detail the architecture plans give the bridge builders.

This means that the design phase in civil engineering doesn't map to solution design in software development – it actually maps to the entire programming effort, which is the bulk of the work, cost and time in any software development project.

        +------------+-----------------------------------+
BRIDGE  |   Design   |              Building             |
        +------------+------------------------+----------+
 APP    |             Programming             |   Ops    |
        +-------------------------------------+----------+

Copying bridges

Ok, you build one bridge. How hard is to build another just like it? Well, you repeat the whole process again in its entirety. Hopefully you do reap some savings due to the fact that you're now more experienced and have established supply chain connections, but that's it. The majority of work is still there and if everything else stays the same, the budget won't be very different.

In software development that's – you just make (or deploy) a new copy. Done!

What does this mean for builders themselves? This means in civil engineering you get more money for building (executing on the plan), while in software development you get more money for developing the solution (the design part). You just don't hear about multinational mega-corporations of architects on a level with FAANG.

Interestingly, the multinational mega-corporations handling ops do exist (PaaS, IaaS, app stores). They combine infrastructure, distribution and operations – in our comparison, in the civil engineering world they'd be an amalgam of builders, utilities and shipping companies all fused into one.

Cost of failure

If a bridge collapses, that's a huge problem, even if noone gets hurt. If software malfunctions, in most cases it's not a big deal (sometimes it is and then it's newsworthy). More often, if it works but badly, people scrape by.

Nowadays you can even fix the bug and push an over-the-air update – no need to recall all the cars of planes you've built with the defect.

This means the cost of failure (either catastrophic, or failure to live up to the expectations) is much lower in software than in either civil or mechanical engineering. That again means the return on investment (ROI) on any effort mitigating that cost is small, discouraging spending time and money on it.

So software today is buggy becase it can be. It's slow because it can be. It's insecure because it can be. It's not due to any lazyness of programmers, incompetence of managers or malice of big bosses. It's how the world works – if you spend too much on quality where not required, your competitor can spend the same money to grow faster with no downside.

Yet it is not obvious that it is unequivocally a bad thing. Consider the failures of famous projects with (near) unlimited budgets and perfectionism, such as Duke Nukem Forever. It makes as little sense to overengineer a software product as it would be for a building. Tech moves at breathtaking speed, and we've come so far in just a few decades, because it often can “move fast and break things”. Software can be more adaptable and quicker to the changing needs.

As software becomes a vital part of more and more things in the real world, the costs of failure rise. Whereas you might be only slightly annoyed at having to restart your Facebook app, you really don't want any trouble from the software controling your two-tonne car while you're doing 80 on a highway.

If that means more software will need to move slowly and avoid breaking things. And maybe we'll be more like the bridge builders.

By this I don’t mean spam arriving in our inbox, viruses or malware lurking in software downloaded from less-reputable places, or phishing sites masquerading as our favorite e-commerce platforms.

These risks are real, but well understood and widely recognized. However, in the past years there has been an increasing evidence for, and occurrence of, completely different kinds of risk that most of us online are exposed to.

Examples of these are pervasive tracking of behavior online, appropriation of personal data by the apps or sites we use, data breaches, and junk media optimized to maximize engagement.

Before I go over each of these in more detail, a disclaimer: I don’t think everyone’s out to get me, or that big corporations such as Google or Facebook are inherently evil. I do think that companies, big and small, are incentivized to behave in ways that create or increase these risks. That is, the default is to behave in a way that makes things worse.

Tracking

Start with tracking. Google and Facebook know every page you visit if it has Facebook or Google login, social or like buttons, embeds fonts or maps, uses Google Analytics or any of their dozens of APIs. So do the ad networks: a handful of major ones are used on most sites, and they track unique users so they can build your profile, optimize ad inventory that you see and retarget you. This means they follow you around the internet to show you ads for products you viewed but haven’t bought yet.

Google and Facebook, the portals to the online world for many, know the most about us. But they are not unique in this regard: companies such as Twitter, Amazon and virtually everyone else does it as well.

Is this really a problem? I believe so. I personally don’t like my privacy being violated at will by a random site I happen to visit. On a practical level, I understand that the companies collecting this data aim to maximize their shareholders’ value, not my benefit. While some amount of tracking is acceptable to improve the service I get — and people may have different notion of what’s acceptable to them — there should be a way to draw the line somewhere instead of going full-in.

Tracking can be countered by using an ad blocker, such as uBlock Origin or AdBlock Plus. Today’s ad blockers do more than just block annoying ads: they also disrupt all kinds of invasive tracking, and can be integrated in all modern browsers and mobile devices. This approach does have a side-effect of blocking ads too, depriving sites of revenue. However, at this point I don’t think browsing the web is at all viable without an ad blocker. To put it bluntly, the experience is horrible.

I also use DuckDuckGo, an alternative search engine with a focus on privacy and usability. Its results are usually slightly worse than Google’s, but it does have a few extra tricks up its sleeve (such as direct searching of a specific sites) and it’s easy to fall back to Google, so it’s tradeoff I’m willing to make. DuckDuckGo also has a browser extension which can also block tracking software and report site’s privacy score, among other things.

Finally, I use Firefox with Multi-Account Containers and First-Party Isolation features enabled. These are “block 3rd-party cookies” option on steroids, completely isolating each site so no cross-site tracking is possible. The side effect is disrupting features such as log in via Google or Facebook, comments or likes, and site widgets from 3rd party sites. Equipped with a good password manager (I use 1Password), I find this only mildly annoying.

On mobile, I use Firefox Focus, which behaves like a browser in incognito mode, making it easy to forget all history (including any tracking cookies) with a single tap.

Personal data

The amount of information big internet giants track about us is dwarfed by the amount of data we freely give them: photos, videos, text posts, travel and purchase information, our plans, intentions, fears and desires. And for the most part, they can keep this data forever, use it as they like, including giving others access to it. This has been somewhat limited by the European GDPR and the series of privacy scandals involving Facebook intentionally and unintentionally giving others vast amounts of what should’ve been private data. But it is still largely in place for those not inclined to, or not aware that they have the option to, micro-manage what rights over their data they give Facebook and other big companies.

The problem here lies in not seeing through the implications of this. When you tell Facebook (or Google, …) something, it remembers it forever. For instance, that embarrassing photo or status update you hope everyone’s forgotten by now. That awkward private message that you sent as public instead. That photo of you six months old naked in a bathtub that your parents thought was infinitely cute and just had to share publicly at the time.

All of this will be used, to sell you stuff or to make you come back for more. If you get embarrassed, mobbed, fired or worse — hey, you shouldn’t have posted it online.

Which brings me to the best way to minimize this risk: treat everything you post as if you’ve shouted it on prime-time national TV. If you wouldn’t be comfortable letting the world know about it, don’t put it online.

The only exception to this is email and private messages. Data breaches notwithstanding, these usually come with privacy implied and companies take care to protect these. But even here, it pays to be cautious because your conversation peers might not be.

Another way to ensure your privacy online is respected is to periodically — say, once a year — visit privacy and security settings of the sites you use and verify that all the settings are to your liking. These companies have an annoying habit of changing available privacy controls which then default to something the company finds useful, not what you might’ve wanted.

Data breaches

Massive data breaches, exposing passwords, social security numbers or other private and sensitive information of thousands or even millions of users, are nowadays a common occurrence.

While perfect security is impossible, the fact is that companies are not incentivized to strive for this perfection. One of the larger data breaches, that of up to 40 million credit and debit card details of Target in 2013, cost the company $202 million in total. This is in comparison with $2.4 billion net income for the company in 2017.

The largest data breach in 2018 was that of the Marriot Starwood customers' data, affecting anywhere between 300 and 500 million customers.

Laws like the European General Data Protection Regulation (GDPR) and California’s Consumer Privacy Act are slowly changing things for the better, but there’s still a long road ahead.

Individually, the best protection is following security best practices such as not using the same password on multiple sites, using HTTPS, enabling 2-factor authentication where avilable, using end-to-end encryption for private messaging, and so on. This decreases the problems you have when (not if) one of the sites you visit has a data breach.

Junk media

I use the term “junk media” for content that’s primarily designed to get eyeballs, not provide useful information, be interesting or entertain. A few examples are textual and video content farms, social media feeds optimized for engagement or viral content, or irrelevant “breaking news”. Again, the line here is blurry and everyone will have differing criteria.

Why am I mentioning junk media in a post about staying safe online? Similar to over-sharing of our personal data, this is something we do to ourselves without really thinking about it. Accumulating over the longer term, it can also have negative consequences for us.

Junk media may be “fun” or “interesting” in the sense that we have an instant reaction, just like junk food can be tasty while containing poor nutritional value. In either case, indulging in moderation is not a problem, but a steady diet of either won’t be good for our health.

The problem is that moderation doesn’t maximize revenue. In purely commercial terms, the winning strategy for the media companies is to maximize views and engagement while minimizing churn. The more time we spend on those sites and the more content we consume, comment on or share, the better. The quality of time spent for the consumer is of secondary importance — just good enough to prevent people from leaving.

Junk media is not confined to online. It’s equally present in the press, on the TV and the radio. In the past, there’s been a lot said of negative effects of too much TV. Comparatively little research has been done into negative effects of too much social media.

Not consuming too much junk media is as easy — or as hard — as not overeating junk food: just don’t do it. A more actionable advice is putting it “out of reach” so you won’t unthinkingly reach for it. For example, I open Facebook from an incognito browser and have 2-factor authentication enabled. This forces me to go through multi-step login process each time I want to visit, making it inconvenient enough that I only visit if I really want to. For the same reason I also haven’t installed a Facebook app on my phone — it makes it too convenient to dive back in.

I’ve titled the post “Digital hygiene”. As with the regular form, digital hygiene consists of small things we can do every day that improve our health and minimize health risks.

Starting with the security best practices, thinking about what kind of information we’re sharing (willingly or not) with companies and the larger public and the possible implications down the road, we can change our behavior ever so slightly to minimize the downsides, while still reaping the benefits, of living online.

This post is my attempt to raise your awareness of some of these things, share a few practical tips, and give you some food for thought.

Universal Basic Income (UBI) has started appearing with increasing regularity in research and experiments all around the world (Finland, India, …Oakland?). Of course, the scheme has both benefits and drawbacks, its proponents and critics, but in the absence of experience from a large-scale long-running UBI program, it is hard to evaluate what would actually happen.

One popular critique is that giving everyone some amount of money would simply raise the floor for prices. Inflation would do the rest and the scheme would quickly cancel itself out.

This critique is actually easy to dispel, and I believe it’s useful to do so, so we can focus on other actual challenges (of which are many). Here we go:

If everyone gets a fixed sum of extra cash, what’s to stop merchants from raising prices? Other merchants. Consider bakeries. They could raise prices, knowing everyone has extra money to pay for bread. However, all it takes is one savvy baker to recognize he or she could raise the prices just a little lower than everyone else, enough for people to start preferring their shop instead of the competition. The baker could increase their market share significantly (open a chain of “cheap” bakeries). But the competition would quickly catch up to the rascal’s plan and lower their prices accordingly. The baker could lower them still a bit more, and so on…

Where would that downward pressure stop? At the point at which there is no point in running the bakery (the profit is too small). Which is the exact same price point as before[0], and doesn’t depend on the purchasing power of the consumers (that is, it’s not tied to how much money people have).

However, in this description I’ve made two important assumptions: that there is enough competition between bakers, and that bread production can be ramped up and down as demand increases. If either of these assumptions is false, the picture becomes less rosy.

Start with competition. If there is only one baker (a monopoly), he or she is always in the position to charge as much as they like (which is typically just below the point at which many people would stop eating bread and switch to something else, say, rice). In this case, introducing UBI would directly lead to price increase, unless the price itself was regulated (as is the case with utilities, which are natural monopolies).

The other assumption is that the production can adapt to the demand. When this is not the case, that is, when the supply is limited, the competition between consumers for a limited number of products, will almost certainly gobble up any extra money people receive. Spectacular example of this is the housing market in Silicon Valley, where an ever increasing number of IT workers with sky-high salaries competes for a very limited amount of housing.

An even better example are tuitions for prestige universities in the US. Since it is not in the universities’ interest to increase number of students, increasing the money supply for the prospective students with student loans meant that students were now able to pay more for the same thing and that universities could simply increase the tuition fee[1]. Increasing the money supply to students via UBI would have the same effect.

Coming back to validity of the critique that UBI would simply result in price increases, we can see that it rests on the question of whether people spend more money for commodity products, or on limited-supply products or monopolies.

The recent stats from US Bureau of Labor Statistics[2] show that roughly a third of the expenses are housing related. To me, this shows that those in very skewed housing markets (like Silicon Valley, New York, or London) might see price increases due to UBI, but for the most people (that live in healthier housing markets) the housing cost shouldn’t be affected. Other costs are related to more commoditized goods and services so they should be even less affected.

This doesn’t mean that Universal Basic Income is definitely a net benefit for society. There are many other issues to examine, challenges to be sorted out, and the jury will be out on its effects for a long time.

But at least we’ve got one out of the way.

[0] Actually, it could be even lower. If UBI replaces minimum wage, workers may decide they’re willing to work for a little less, thereby reducing the cost of bread. I’m not an economist, statistician, or a social scientist so I will not venture into discussion on whether that’d be a good thing overall.

[1] That’s not to say student loans weren’t beneficial overall. It may very well be that the system allowed more students to attend the universities as not all schools’ prices hiked (and not nearly by the same amount as the top ones), and allowed more middle-class and poorer students the opportunity. I know too little about the matter to draw any conclusions either way.

[2] I imagine stats for other western countries would show qualitatively similar amounts.

Useful voice recognition, combined with AI capable of parsing specific phrases and sentences, is finally here. Amazon’s Alexa, Apple’s Siri and Google’s Assistant are showing us what the future will be like.

However, the safeguards are lagging behind the capabilities, as the recent example of a TV anchor ordering dollhouses shows. The fact that the system picked up voice from the TV and interpreted it as a command sounds funny, but should be terrifying to anyone remotely interested in computer security. It sounds like a Hollywood adaptation of the classic remote code execution bug — but it’s not a fantasy any more.

We’re so happy that we have machines that can listen to us, that in our rush to use / buy / create them, we haven’t stopped and made sure they listen only to us. That’s why a kid can order a dollhouse while parents are asleep or away, TV anchor reporting on that can order hundreds more, and we can play fun pranks when visiting friends by ordering tons of toilet paper while they’re not looking :–)

Accidentally ordering something online can be terribly inconvenient and cost you a fine buck, but as these assistants get control over more devices in our homes and our lives (IoT anyone?), we’ll start seeing real problems. Here’s a stupid trick that might just work in a year of so: Alexa, unlock the front door!

Mobile phone voice assistants show one way of handling this: by requiring the phone to be unlocked for (most) commands to work. Yet while may make sense for phones (and only slightly inconvenience the user), it’s a non-starter for home automation systems. If I have to walk over and press a button, I might just as well do the entire action (such as turning the light off, or unlocking the door) myself.

Another possibility is speaker recognition. By analyzing how the words are uttered, not just what they are, such systems can distinguish voice of the authorized user. However, like many other biometric systems, it is easily fooled by a facsimile of the user — in this case, a simple recording. Thus anyone with a mobile phone can “hack” this kind of security.

More effective, and only slightly more inconvenient, would be the combination of requiring the physical presence of the user in the room (for example, by sensing their mobile phone, smartwatch, or other personal item they’d carry around most of the time) and speaker recognition. In this case, even if a hack is attempted, the user themselves would be around to prevent it.

So the good news is, it shouldn’t be that hard to build more secure voice-controlled systems. The bad news is, as we’ve seen with huge botnets made of compromised IoT devices, many companies in home automation space currently have no experience or incentives to focus more on security.

Voice-controlled AI assistants are here to stay, and it’s a good thing — they’re mightily convenient. But expect more fun anecdotes and scary stories in the years ahead.