Experimenting with static site generation

Hi!

If you visit this blog on a regular basis, which you probably don't, you may have noticed that it looks a bit different from how it used to look. The overall theme is the same, but the URLs look way different, and there is no comment section. So what happened?

A while ago, I wrote my own content management system in PHP, replacing WordPress. That was your standard CMS, for every request, you have a script (PHP script in this case) which generates the page dynamically, showing you the content you requested. March this year (2014) though, I started a new project, which works fundamentally different.

Enter jsSiteBuilder.

jsSiteBuilder is a static site generator. That means that nothing is built on the fly. There's a script which reads the content of a MySQL database and generates all the required HTML files for each and every page and post. Whenever a user requests the website, the one and only thing the web server ever does is what web servers do best - it fetches the file and sends it back to the user.

The most obvious advantage with this method is performance. Both server load and request time is brought down to the absolute minimum. It also provides some much needed stability. Your website doesn't go down, even in the case of an extreme disaster. Say you accidentally delete your MySQL database, or the database host goes down, or file permissions mess up. Usually, this would take down the website. With this CMS however, all html files will just stay there, available for everyone to see. Your users won't notice a thing, while you can take all the time you need to properly fix whatever issue appeared. You can even re-run the site building script as much as you like while everything is down; it won't delete anything.

Not everything with jsSiteBuilder is static however. More specifically, it has the admin control panel you'd expect from an old-fashioned dynamic site generator, like WordPress. That's because the admin panel is written in PHP. This makes it easy to create, edit, and generally administer and set up the website. All the admin interface does is to interact with the MySQL database. Once you're done making whatever changes you want to make, you can update the user-facing portion of the website with the push of a button. On my blog, with my server, completely regenerating the entire site takes no more than a few tenths of a second.

Read More

A small WTF regarding CSS units

CSS. A tool loved by web developers all over the world. It lets us style our HTML easily, and creates somewhat loose coupling between content and layout.

CSS. A tool hated by web developers all over the world. It makes us spend countless hours trying to accomplish what seemed to be the most mundane task.

Today, CSS got a little bit weirder.

It all started with me and a discussion between me and a friend, Stef Velzel (or Invalid). He has a website at ckefworx.com, in case you're reading this in the year 3026 and he has finally removed that "under construction" banner.

Anyways, Invalid and I was discussing, as we so often do. I had come across a blog post on Reddit claiming that in CSS, one pixel (px) is always 1/96 of an inch. I even found CSS specifications, which supported the statement. Invalid disagreed though, and meant that 1px is always one pixel on the screen, according to his experience.

It turns out I was right. In a sense at least. But so was Invalid. You see, 1px being 1/96 of an inch and 1px being exactly one pixel isn't mutually exclusive. Not with CSS at least.

"But wait", I hear you say. "That doesnt make sense?"

Well, yes it does. Sort of. If you're a mathemagician, you may have figured out this already. The only way this adds up, is if one inch is 96 pixels, 96 points of light on your screen.

Have you spotted the issue here? no? yes? 96 pixels isn't equivalent to 1 inch. Not in a world where pixel density varies wildly from device to device. CSS can't just redefine inches like that can it? I mean, 1 inch is exactly 1 inch, isn't it..?

Well apparently, CSS can indeed just redefine units of measurement like that. 1 inch is 96 dots of light, not 1 inch.

On some deep level, this makes sense. It seems like common sense define units of measurement out from the fundamental unit of the display instead of arbitrarily defined real life measurements. The problem is that it's marketed as inches, centimetres, etc. instead of what it actually is. This causes a lot of confusion.

I should probably add that the spec doesn't state that one inch should be 96 pixels. It rather says that one px should be 1/96 real world inch, which at least makes a little sense. Browser vendors implement it how I described above though, and in the end, that's what matters.

Update: I should probably have included some of the tests I did, and some sources.

First off, let's see how a pixel is defined by the W3C:

The absolute length units are fixed in relation to each other and anchored to some physical measurement. They are mainly useful when the output environment is known. The absolute units consist of the physical units (in, cm, mm, pt, pc) and the px unit:

cm: centimeters

[...]

in: inches; 1in is equal to 2.54cm

px: pixels; 1px is equal to 1/96th of 1in

[...]

According to that, it would seem like 1 inch is exactly 1 inch, regardless of resolution. 1px should arlso be the same regardless of resolution, as it is defined using inches.

Look at this example element however:

<div style="background-color: #000; height: 1in; width: 1in"></div>

I don't know about your browser, but mine does at least not render that as exactly 1 inch. Have a look at this however:

<div style="background-color: #00F; width: 1in; height: 1in; display: inline-block"></div>
<div style="background-color: #F00; width: 96px; height: 96px; display: inline-block"></div>

I don't know with you, but those look fairly similar with all browsers I've tested with. This shows that, regardless of what the CSS spec says, an in isn't an inch, at least not in all browsers. It's 96 pixels.

Read More

Rant about YouTube

Since its inception in 2005, YouTube has grown out of proportions. It is to videos what Google is to search.

With a user base of billions of people and thousands of hours of footage uploaded daily, you'd almost think they knew what they were doing. And they do, from an infrastructure standpoint. That amount of traffic requires vast server farms all around the world, all working together.

Where YouTube lacks however, is in terms of its user interface. There are some real disasters in this department.

Take for instance selecting a video's quality. When you change the quality from the default 360p to say 720p, and user would expect the quality to change. The user model says that when you change quality, the quality changes. Believe it or not, the YouTube team actually managed to get this wrong.

When you change a video's quality, the video remains in its original quality. It changes to the selected quality only when it has played through all that is already buffered.

For me, a pattern like this is not unusual:

I click on a video. I fullscreen it on my big 1080p display, before changing the quality to 1080p, but due to my 70 Mb/s internet connection and YouTube's great infrastructure, 3/4 of the video is already loaded. I proceed to watch 3/4 of the video in horrible quality, before it switches over to beautiful full HD.

One definition of great software, is that the program model corresponds to the user model. Basically, the program should behave like the user expects. The user model is definitely not that changes in quality settings applies only after watching a random portion of the video, if at all.

Another cause of bad usability is frequent changes in the user interface. Not small changes, like altering the looks of a button here and adding some gradients there. No, we're talking total overhauls of the UI. Completely revamping how everything is structured.

YouTube has had quite a few of these huge overhauls. Some times it has restructured everything multiple times per year.

Users hate big changes. The reason is that users don't analyze the interface and look for the most logical place for a feature every time they need said feature. No, we users memorize where what we need is, and navigate there out of habit more than anything else, at least with programs we use frequently.

When the user interface is restructured, we still go looking for what we want in the areas we're used to. When that doesn't work, we lose our feeling of control. The program now has control over you. This happens unconsciously, and leads to frustration.

Furthermore, it forces us to reanalyze the interface and look for features we previously knew where was. As it turns out, this causes quite a bit of cognitive overhead. Our brains are way better at just looking up already stored information than processing brand new data. When YouTube us completely overhauling its website multiple times per year, this becomes quite a bit of an annoyance.

The mobile application for iOS, and possible Android, is a bit of a disaster too. At least in sone respects. Sure, it has its bright sides, but as this is a hateful rant, I'll jump gracefully over those and focus on the bad aspects.

In YouTube, there is a comment section, as you may know already. It's possible to reply to people's comments. You can even see who a comment is a reply to, and by the press of a button, you can see the original comment.

The team working on the mobile app rightfully decided to implement the comment section. What they did not however, was to let you see who a comment is a reply to, rendering it useless. You see, this reply-to-feature is frequently used. It's used so much that without it, the comment section is just a bunch of random statements, completely out of context.

The app also has a feature which dynamically sets the video's quality according to your internet speed. Unfortunately, this doesn't quite work right.

In my home, there's an area which the router's WiFi doesn't quite reach. When I move into it, YouTuve notices the bad connection and drops the quality to below unbearable. It seriously looks like a grunge teared apart by a crack in the space time continuum, just slightly less fancy. Unfortunately, it does not adjust itself once the connection picks up again for quite some time. This forces me to exit out of the video, scan the list of videos for the proper one, wait for it to load, and navigate to where I was. As you might expect, this completely kills my flow.

In short, how to fix YouTube:

  • Make quality changes immideate
  • Less frequent UI overhauls
  • Implement replies into the mobile app's comment sections
  • Provide optional quality controls to the mobile app
  • Make the app better at automatically selecting quality
Read More

JavaScript's Rough Childhood

As some of you may know, I'm a fan of JavaScript. Pretty much all of my projects are web apps, and as such, JavaScript is an important part of them.

JavaScript does have its good parts. Parts which at the very least are en par with other languages. It does, however, also have quite a bit of bad parts.

In the beginning, there was Brendan Eich. Eich got hired by Netscape to design a programming language for their web browser. At first, what he had in mind was something resembling Scheme, a dialect of Lisp.

When he had worked for some time on this Scheme-esque language, someone, presumably in Netscape's management, decided they wanted something else. Eich got told to start from scratch. This whole Java-thing seemed to be taking off, so he would have to make it more like Java. "And by the way, we need it in ten days", they told him.

Now, ten days is orders of magnitude less than you need to design a great language. Eich did a great job, but as you would expect however, it did have quite a few unforeseen quirks.

Unsurprisingly, Microsoft decided to copy Netscape. They had a team dedicated to find JavaScript's quirks and replicate them.

Fast forward a bit, and Netscape submit their language to European Computer Manufacturers Association, or ECMA, to make it a standard. ECMA agrees, on the premise that it won't be called JavaScript anymore. A team of people started writing detailed documentation for what was internally called ECMAScript. Microsoft had a key role in this documentation process, and due to their work on accurately cloning Netscape's JavaScript implementation they knew exactly what odd quirks JavaScript had, and thus what they should make sure to avoid. Or, as it turns out, make sure it got into the standard. Yeah.

This glorious work from Microsoft's side is part of the reason JavaScript is the inconsistent mess it is today. Take for instance how typeof null is "object". That, and a bunch more, is a result of the incredibly short amount of time Eich had to make JavaScript, and Microsoft's effort to make sure all quirks from the original JavaScript implementation stuck in the ECMAScript standard.

Abstraction

Even though it can be tempting to blame Microsoft for everything wrong in the world, it should be said that they aren't the root of all problems with JavaScript. Some of the problems aren't even real problems, but a result of leaky abstractions.

It is fairly obvious that JavaScript tries to be fairly abstract. Take for instance how it doesn't have types. That's an abstraction. Internally, the computer does distinguish between text, integers, numbers with decimals, booleans and more. The JavaScript language tries to hide this however. In the world of JavaScript, everything's a "variable", declared by the keyword var.

One problem which almost exploded in my face is related to how some of JavaScripts types are passed by reference, and others by value.

Now some of you may never have heard of passing values by reference or values is. Nor do you understand why it's a big deal. Even if you're a programmer, this can be a completely foreign concept for you. If that's the case, chances are you're using a very abstract language like JavaScript.

I won't get too much into the inner workings of computers, but I will try to explain the basics of passing by references and values.

Say you have two variables, Foo and Bar. Say you set Foo to 5:

Foo = 5

Now, we set Bar to Foo:

Bar = Foo

If we pass by value, Bar and Foo will be two distinct, completely unrelated variables. Changing one will never ever in a billion years affect the other.

If we however pass by reference, Bar isn't a value in itself. When someone ask what Bar is, it simply says "go check out Foo, maybe he knows"; Bar is what we call a pointer. Because of this, when you change Foo, Bar's value also changes. If we set Foo to 10, Bar is also set to 10. This also works the other way around, at least in JavaScript. If we set Bar to 12053, Foo will be set to 12053. The two variables are the same, just under different names.

Passing by reference can be a lot faster when dealing with big variables. Therefore, JavaScript passes some types by reference. Those types are functions, arrays and objects. The problem here is that when programming in JavaScript, there's no clear distinction between types. After all, that's the point of being an untyped language isn't it?

This can cause some really confusing quirks. For instance, after this code, bar is 10:

foobar = 10;
bar = foobar;
foobar = 20;

while after this code, bar.val is 20:

foo = {"bar": 10};
bar = foo;
foo.bar = 20;

If you're not experienced in JavaScript, or programming in general, this might not make a lot of sense to you. Trust me though when I say that this can cause severe problems.

Of corse, the whole problem would be gone if JavaScript by default passed values by value. Passing by reference could be an option. This is how C does it, and it works great.

There are other quite freaky abstractions out there. Take for instance how JavaScript doesn't force you to use semicolons at the end of lines. It does this by automatically inserting semicolons where they are missing. One of the ways it does this is really creepy: it runs a line of code, and if it fails, it inserts a semicolon at the end and tries again. There are a few problems caused by this which I wont get into, but most of all it's just creeping me out to know that JavaScript does that. It does also teach new programmers the horrible custom of ignoring semicolon, so using languages where semicolons are required becomes a hell. Therefore, use semicolons!

Solutions

What can we do to make our web programming lives easier, and overcome JavaScript's flaws?

One of the solutions can be to translate other languages to JavaScript. Lots of such translators have been made, and nowadays you can translate pretty much any language C, C++, C#, Java, you name it into JavaScript. People are even designing languages whose sole purpose are to be translated into JavaScript code. CoffeeScript is an example of this. A problem with translating other languages however is that the web browser will still be running JavaScript code, and will spew out errors in the JavaScript code. It can't magically know where in the code you wrote the error is. This adds a lot of complication to debugging, and you pretty much have to be fluent in JavaScript anyways to be able to see what the error really is.

Another solution is to simply go with JavaScript, learn to love its quirks, or at least learn how to overcome them. Know that when you typeof null, it will return "object". Learn that if you declare variables certain ways, they are objects, arrays or functions, and as such are passed by reference, and learn what passing by value/reference really means. Learn to always have your Google machine ready. Learn that while high levels of abstraction makes languages a lot easier to get involved with, it also makes it quite a bit harder to really get to know the language.

(some of the things I've written here, I learned from a talk about JavaScript. I think the talk was by Douglas Crockford. Sadly I can't find it again.) As some of you may know, I'm a fan of JavaScript. Pretty much all of my projects are web apps, and as such, JavaScript is an important part of them.

Read More

"'Considered Harmful' essays considered harmful" essays considered harmful

Okay, that title is a bit of a brain twister. Hear me out though, I promise I'll eventually make some kind of sense.

Since the late 60′s, a type of computer-related essays, namely so-called "considered harmful" essays, became popular.

Considered harmful essays are all about writing page up and page down about why something programming related is bad and should be avoided. The first considered harmful essay, at least the first somewhat mainstream one, was written in 1968 by the Dutchman Edsger W. Dijkstra It was called "Go To Statements Considered Harmful", and, as you might have guessed by now, is about how GoTo statements have a tendency to produce some really messy spaghetti.

After Dijkstra's essay, the style of writing got so popular you could say it became a clich. We got a metric ton of "considered harmful" essays, each essay nitpicking on its own small area. With statements, XSL, Java, the "new" keyword, namespaces - all of which, and more, considered harmful by someone or another.

One of the later additions to the considered harmful family of essays is "'considered harmful' essays considered harmful".

Now, what's harmful about "'considered harmful' considered harmful" essays?

Well, one of the more obvious effects it has, is this post. I mean, "'Considered Harmful' essays considered harmful" essays considered harmful. If that title doesn't blow your brain out of your ears, I don't know what will.

But it doesn't stop with molten brains. Oh no, far from it. You see, considered harmful essays aren't really necessarily there to tell you not to do or use whatever the essay is about. It works more like a warning. Not as much "don't do x", but "before you do x, make sure you know what you're doing".

There are lots of new programmers out there. I myself am fairly new. With the extreme levels of abstraction in the languages which are considered great for beginners, it's easy to do something which in the code looks completely sane, but when a virtual machine runs it, it forces the CPU to do a gazillion operations. If you had taken a slightly different approach to the problem however, it would only have taken a few billion operations.

Or things could behave unexpected. For instance, the language could suddenly decide that nope, that variable (everything's variables these days - goodbye data types) is passed by reference, while all other variables are passed by value. This can have gastronomical implications, and completely break a project.

Many considered harmful essays are there to tell you about those pitfalls of leaky abstractions.

In addition to the informative value, they're a joy to read. I myself do at least love reading a well written considered harmful essay.

Read More