natural treatment of hypothyroidism

progr.es Behind the Screens – Part 1

An overview of the progr.es technology by Herman Schaaf, lead software engineer.

This forms part of a two-part series on some technical aspects behind progr.es. I will try to give some insight into how we chose the technologies that we use, and how we managed to make these technologies work well together and perform some of the intricate features of the main progr.es application in a simple, efficient way. The latter will specifically be discussed in Part 2: Gluing it all together.

Choosing Between Buzzwords

In the building of progr.es, we had to make a number of technology-related decisions. Most importantly, we had to choose only a few core technologies – from the swarm of buzzwords that invade our daily vocabulary – to power and assist in the development of progr.es. I need only throw a few names out there to give an idea of the overwhelming number of choices even when merely deciding on a backend application framework: Django, Pylons, Flask, Node.js, Express, Ruby on Rails and Scala are merely a few examples. In evaluating every framework, we gave careful consideration to a number of factors, before settling on our rather unique stack.

Firstly, we evaluated the framework on a high level to see what advantages it offered for our particular project. Node.js, for example, comes with socket.io which makes it easy to have immediate two-way communication between the server and users who are currently using the site. Other libraries offer similar capabilities, but none quite as developed and impressive as socket.io.

Secondly, an important factor was the amount of experience the team had in using the framework. An experienced developer shouldn’t have trouble adapting to a new environment, but even so, development will happen more slowly than when the team is working with frameworks they know well. When using a new framework, more problems are likely to creep up due to inexperience, and bugs will take longer to resolve. This will be the case for at least a couple of months, and the project’s time constraints will therefore necessarily play an important role in deciding whether or not to jump in with this new framework, even if there are other advantages at play.

Thirdly, we looked at the maturity of the frameworks. In terms of the size of the community, the length of time it has been around, the momentum behind it at the moment, and the availability of help and answers to questions. With all these things should also come an assurance that the framework will still be around, updated and popular three or four years down the line. This insures ease of maintenance and sourcing new team members in the future.

At the end of the day, given enough development time, any framework can perform any task we may throw at it. The question was therefore not necessarily whether we would be able to build progr.es using a certain combination of frameworks, but really how much of our own development time it would take for our product vision to reach completion. In developing web applications, time is the commodity that puts you ahead of the competition. If you can just plug in a library to do what you need with minimal development time and your competition is writing that library themselves, they will be spending a lot of extra time and money building their application, while you are already getting users and refining yours.

Our Platform

With these factors in mind, we settled on a stack combination that is unique to progr.es and truly powerful in the way the technologies complement each other: Django/Python for application logic; its maturity as framework, its vast array of plugin libraries and my own experience with it being the biggest factors in the decision. Django has a couple of real-time communication libraries, but no python library impresses as much as the JavaScript-based Node.js and socket.io. We therefore decided to use those two purely as a transport layer for instant messages to and from the client. How that works in combination with Django, I’ll explain in more detail in a follow-up post. For storage we decided to go with the fast-growing non-relational approach, and MongoDB seemed like a good and mature fit, with the Python-based mongoengine a good library for abstracting communications between Mongo and Django. With progr.es projects being relatively self-contained units, we rarely require complex queries that a non-relational database cannot deliver with much better speed and scalability, so the choice was really only which non-relational database to use. Finally, on the front-end, we went with the fairly traditional stack of HTML/CSS/JavaScript with jQuery for rendering and updating. There were other options to consider, like Angular, Backbone, Knockout and Ember.js, but we decided to not over-complicate things at first. In the near future, with our JavaScript getting ever-more complicated, we may re-evaluate using some of these libraries.

Our chosen stack is powerful in that it manages to supplement the short-comings of each framework with the strengths of another. This is especially true for the unusual combination of Django and Node.js: where Django falls short on real-time communication, Node.js picks up the pieces and provides a beautiful layer for JavaScript-based communication with the client. At the same time, our team (at the time) did not have great amounts of experience in using Node.js, and so by not writing the entire application in Node.js, which would also have had other disadvantages, like losing useful Python libraries, we could offset the time-risk associated with jumping in with something completely new and have the sturdy backing of the relatively mature Django framework.

Buzzwords are just that, buzzwords, until one goes into the trouble of reading deeper and researching each for its true merits. Then they start changing from buzzwords to tools that help you get certain things done faster. Sometimes these tools feel like using a power drill for the first time, when you’ve only ever used a screwdriver. All in all, we are truly satisfied with the framework choices we made: it helped us build a high-quality first product iteration in just a few months since conception, and believe it will allow us to iterate faster and build smarter for a long time to come.

In Part 2, I will discuss how we managed to glue all these frameworks together from a technical perspective.

Herman Schaaf is the lead software engineer for progr.es - you can follow him or ask questions @ironzeb on Twitter.

Google+