The Entropic Web
8th December 2019
In the online world, chaos reigns supreme, and its the job of technology professionals to bring order to the ever entangled place that we call the web.
In this one, we are getting down with thermodynamics, the internet and the notion that everything tends to disorder.
What is Entropy?
Entropy is a thermodynamic quantity representing the degree of disorder or randomness in a system. The second law of thermodynamics is key as it states that entropy always increases with time.
To understand entropy checkout the video below where Brian Cox uses sandcastles to make sense of it all.
So what’s this got to do with the Web?
In the context of the web, you can think of the sandcastle example as the internet in its state today, a structured and somewhat ordered low entropy entity. An entity ready for search queries thanks to search engines.
Keeping search in mind, each day the web grows with new sites, pages and applications being added to it. Like the sandcastle, the additions are akin to extra grains of sand, providing more disorder in the structured version of the web.
Ultimately this makes it become less structured once again. But its the job of search engines to bring back order and manage our neatly structured and searchable web, so we as humans can find what we need.
That said, each page added needs to be analysed, which takes up more and more of a search engines time and increases the overall pool of data to search across, thus making it harder to order. Hence the phrase…the entropic web.
On top of that you can ask the questions like, what is the ordered web anyway? Is it fair? Does it have a bias? Does it in-corporate paid ads and should it be allowed to aid monetary agendas?
Personally, the web should be open, but at its core…always cater for the users needs. That way search itself, should provide a view into the ordered web and ultimately be beneficial to the user…with and without advertising. Just as long as it is relevant to the user. But that’s just me!
All that aside, this isn’t even touching upon misinformation and erroneous content. It’s a lot for a search engine to take on! But yes we do have search engine optimisation and conversely we have the opposite of SEO too! Soooo yeah.
These thoughts for me, raise many questions in the area of search…but hopefully you can see, when it comes to getting order online, its chaotic, its hard and there is a lot to be considered!
With that let’s move onto the bulk of this post…as everything tends to disorder anyway, what are we as technologists supposed to do?
Engineering for Order!
It’s simple really and if you think about it, that’s all we are doing anyway. We just need to bring order.
Forgetting online search for a second and just thinking about generic software we produce…for every core customer problem we face, you may find that the current process is either non existent or a bit of an inefficient, clunky nightmare. The end result or the “solution” once the engineering and product development work is complete, is a system that usually provides some sort of order to the chaos, all come about through understanding and solving a customers problem.
Example solutions in the Market
Examples in the market right now that I think sum up engineering for order is Stripes payment integration system, Twillos communication system and Anchors podcast platform, of which this podcast runs through.
Stripe took the trouble of payment integration and said here you go, we’ll do that for you. So you don’t have to. Thank you very much.
Twillo took care of making and receiving phone calls and text messages using its neat web services and APIs.
And the final example, Anchor took the pain of making and submitting a podcast to multiple podcast distribution centres and turned it into a streamlined all in one podcast creation, distribution and monetisation solution. Nice!
So as you can see, all of these examples, showcase the fact that when it comes to the online world, no matter what piece of technological brilliance you are working on, just like in the real world, in the digital one…all we are mostly doing is trying to bring order to chaos.
I mean sure, over the course of our lives, we create different applications and services…but fundamentally it’s all the same, we are always trying to keep chaos at bay and bring order to our world.
In the case of search however, dealing with misinformation, monetary agendas and the sheer volume of searchable content is a tough one. Yes, search engines do a pretty good job at finding what we are looking for. But not everything is “perfect” and this battle is on going and will continue, not only in search, but with most customer problems we face.
The only thing for technologists to do is keep calm and carry on fighting against the ever veering movement toward disorder, in the place we call the entropic web.