What goes into building a site like ShinyThinks?
Interesting question, totally-real-reader-that-isn't-just-me. Depending on the site and who you ask, you're likely to get an answer varying from 'give my cousin a day or two' or 'I need a team of engineers and 3 years.' It all depends on scale, minimum requirements, and what resources you have available.
While I may not be able to categorically answer about a site like ShinyThinks, I can talk about what ShinyThinks is built on.
After we had decided we wanted a site for our projects, we needed to start thinking about a few things:
- Name/Brand
- Content/Audience
- Platform
While Miranda tackled the name and what content we wanted, I started getting the platform set up. At it's core, ShinyThinks relies on the following to run:
- DigitalOcean VPS running Ubuntu
- Cloudflare DNS + other controls
- NameCheap Domain Registration
- Django Python Web Framework
- Gunicorn Python WSGI Server
- Nginx Web Server
- Postgres Database
There are other components I could potentially list like a firewall, python virtualenv, ssh etc, but those are dependencies of the core components or not actually necessary for the platform. Lets talk a bit about why I chose these particular options and what they do.
DigitalOcean VPS - Ubuntu
So there are actually three separate decisions here: web hosting provider, web hosting type, and server distro. I primarily chose DigitalOcean because I already had a server with them and during that research I had come up with them as my preferred provider. When choosing a web hosting provider there are a few things I look for: support, general reputation, tools, performance, and of course pricing.
On both the support and general reputation from DigitalOcean has a great offering in my opinion. Many popular providers are actually under a single parent company that, from what I understand, have mostly combined and outsourced their support and are not generally well-regarded. Whereas DigitalOcean's catalogue of detailed tutorials alone make them my first choice. The majority of this site was actually set up following their instructions.
I haven't had to delve too deeply into their toolset offerings, though the list of tools that are available is reassuring if this site (or another I have with them) takes off and I find myself in need of a more scalable solution. The parts that I use are the monitoring, the easy droplet building, and occasionally the remote access. All fairly standard offerings, but I'm fond of the usability of their implementation.
I've had no complaints about their performance, it's exactly what I'm paying for. On that note, I think they have one of the better small scale prices availabe for low-traffic sites which is what I'm using. Should I need more, I can easily just bump up to the next tier.
So that's why DigitalOcean, but what a VPS? Most simply because I want to be able to manage the server myself. This site is partly to hone my own skills, to showcase my own ability to run the server, and to actually show off the content we are putting on it. Having access to a full server gives me a level of customizability that I would be hard pressed to find on a shared server. Additionally it ensures that other sites do not have a performance impact on this site.
Ubuntu almost speaks for itself. It's probably the most popular linux distro and widely used. Linux is almost universally used for web hosting so that was barely even a decision. Personally I chose Ubuntu because I am the most familiar with it and the community behind it. Many tutorials, resources, and applications are specifically written for Ubuntu so I waste less time translating from another distro to the one I'm using.
Cloudflare DNS
Anyone who has a website has a DNS provider. While many use the same as either their domain registration or their web host, I chose Cloudflare. Cloudflare's free tier quite simply does everything I need it to, plus more, all while being backed by one of the biggest names in web. Their DNS offering is relatively new, but I've had no complaints.
I'm using their always https to redirect http traffic to https (which while easy to do in nginx as well, is a nice-to-have), their cache to ensure static content is snappy to load, and their DNS which has servers worldwide and is consistently top ranked in speed. Plus it comes with mitigation options in case the site comes under attack. Again, keeping in mind that I am using this site to familiarize myself with players in the enterprise market, Cloudflare is an attractive choice.
NameCheap Domain Registration
Quite simply, they live up to their name. They offer solid prices and free WHOIS protection so I don't get spammed non-stop any more than I already do from bots scraping address, phone, and email addresses (made that mistake with one of my early sites, and it's not an easy one to take back). Some registrars have shady reputations around buying up domains that are searched on their sites and then hiking the prices, and I haven't had that concern with them.
Django Web Framework
Here's a big one. Django is the web framework responsible for everything you see here. I knew this was going to be a blog + blog-like project site primarily, but I also wanted the freedom to customize it as I saw fit. So I needed a CMS (content management system). I've tried to build a site built on React as a learning tool and found that I need more time to learn it then I wanted for this site. While one of the most useful in getting a web development job, it would have been a questionable choice for this site due to my lack of exposure to it, needing to build everything from near scratch, and the fact that there are other frameworks more tailored to my needs and skills.
So that's why not just React or Angular, but why not WordPress? It's the most popular for blogs. It mostly boils down to the fact that I am already familiar with WordPress and would not learn as much, it uses PHP which is not exactly my language of choice, and while it supports many, many plugins many of them are also not well-maintained.
Maybe not the most satisfying answer, but it's the one I have. Django has great performance, uses a Model View Controller structure (which I wanted to get more hands-on experience with), lets me build a custom site while still offering many tools of a CMS with how easy it is to get the admin interface to add the actual content. As a bonus, Python is one of the most popular and loved languages around and I have experience with it. It has granular permissions built in, ways to add data, scales, many plugins, and allows me to build custom templates in a language I am both familiar with and enjoy using.
Gunicorn
First off, what is Gunicorn? Gunicorn is a Web Server Gateway Interface implementation. WSGI is a python standard for interacting with web applications. Basically it ties Django into something your browser knows how to display. For WSGI, there are several choices, but Gunicorn and uWSGI are the two associated with Django most often.
I began using uWSGI because it was in the first set of documentation I was following, but it became a hassle to configure and the instructions I was following were outdated. DigitalOcean had a good set of instructions for Gunicorn and it seemed to be well regarded so I went for it. The second time around setting up the Django app for this site went much smoother. Don't mistake that for thinking it's 'simple' or a lesser option however, for example Instagram (the most popular site using Django) uses Gunicorn. It is an excellent option and one I am happy with.
Nginx
Nginx is one of the top two names in linux web hosting, along with Apache. I started out using Apache to serve static sites, but swapped to Nginx in the last couple of years. Originally I did so for it's easy reverse-proxy capabilities (to redirect certain subdomains to different servers in my smart home project which I will write about another time). This is mostly preference, but again the combination of personal experience and DigitalOcean's guides for Nginx/Django won out. Nginx vs Apache is something that many, many other people have written on far more in depth than I can and I'd encourage anyone deciding to look into it themselves and decide which is a better fit for their use case.
Postgres
Postgres is the SQL database that holds all of the data necessary for running this site. At this point I'm likely going to start sounding like a broken record, but Postgres vs MySQL/MariaDB/MSSQLis a topic that has been endlessly debated and you'll find better resources on the topic elsewhere.
For ShinyThinks, MSSQL was out because I wanted linux, open source and no licensing issues. Postgres has all the capabilities I need (and thousands more), far greater performance and scalability than this site will ever need, and I am most familiar with it among the open source options. On a side-note, Django comes pre-configured to use SQLite for development so you can get up and running without needing a standalone database if you are just trying it out for test / learning purposes. But I don't think anyone would recommend keeping it that way in production.
And that's all! All the core tech behind ShinyThinks as a website and the decision making process behind them. It's always an interesting process drawing the line between learning something new (Django) and getting overwhelmed with too much at once. Building a site in a new framework was already a potentially daunting task (though I will say Django has been a pleasant surprise to work with on that front) but I am very happy with the results thus far.