Why Is My App Slow? Part 3 - Extensive Data Processing

This is the third article in my series about typical scenarios when a web or mobile app experiences severe slowdowns, causing frustration for both users and the owner.

In the first article, I discussed a project where engineers didn't follow proper programming practices and kept endlessly copying pieces of code, thereby depleting the system's processing power. Experienced programmers easily solved the problem by simply removing all these duplications, which seemed like some kind of magic to the customer.

The second article discussed the same app but in a completely different scenario - a surge in user numbers over several years has pushed the system's resources to their limit. The solution here was again to simply refine the code, although finding additional opportunities to do so was not that easy.

The series of stories would be incomplete without the third one I'm about to share now. In this story speed issues did not show up unexpectedly. We were aware this would happen even before we wrote the first line of code. It was to be engineered as a big data project. That was the customer's selling point. We had to constantly consume tens of thousands of data sources.

Thinking Through

Right after the first deployment to a demo server, the project started to ‘breathe’. One could not leave it for some time and find it in the same state a few days later. In a few days the database would be different - with many more records, restructured and labeled.

And the customer always wanted more data.

It was a responsibility, wasn't it? Some of our developers were a bit scared at first.

And once again, we made a decision to start with simple tools such as Postgres, make proper data normalization, and maintain correct indexes.

We had a powerful production server, but we also used a smaller one for staging with the same amount of data because we wanted to see problems earlier than they would appear in a live environment.

There was a point at which we had to reassess our indexes and queries - but it was done as part of the routine without slowing down the feature development.

It is worth mentioning that if we wanted some special tools, we would have them. We knew about possible solutions like multiple databases and cross-database querying. But we thought that we would always be able to address them if smart engineering was not sufficient.

However, it was. Really, it is amazing how much one can do just by thinking things through and working solutions out. Even with large databases.

Of course, sometimes using special tools is beneficial. And maybe we will get to them in this project one day. But selection of the right tools also requires deep knowledge of the problem to be solved and the way the tools work.

Because tools are just tools.

And good engineering is what makes miracles.

Conclusions

In all three stories I dealt with issues that looked the same way from the outside. They looked like the application took too much time to respond.

However, the reasons for the speed problems were different in all cases. And one would need to look closely into the internal processes, read the logs, use tracking tools, and perform analysis to find the cause.

Well, maybe the first case stands out because the problems were literally everywhere. I could make the application better just by playing around with the code.

What one can learn from that: there is no magic cure that can turn a slow application into a fast one. You cannot just get better hardware or a different technology and expect the app to become faster.

What is more important is how the app is built. If your app is slow, let your developers find the weak spot and make it better. If they cannot do that, find those who can.

For a project owner, it is that simple.