Docker

Docker is almost the rage these days. If you don’t know what Docker is, you should head on over www.docker.io. It’s a container engine, designed to run on Virtual Machines, on bare-metal physical servers, on OpenStack clusters, on AWS instances, or pretty much any other form of machine incarnation you could think of. With Docker, you can easily create very lightweight containers that run your applications. What’s really stunning about Docker is that you can create a lightweight container for your application in your development environment once, and have the same container run at scale in a production environment.

The best way to appreciate the power Docker puts at your fingertips is to try it out for yourself. If you wish to do it, I would recommend the browser-based interactive tutorial on Docker’s website.

While it is easy to build Docker containers manually, the real power of Docker comes from what is known as a Dockerfile. A Dockerfile, using a very simple syntax that can be learnt quickly, makes possible automating the process of setting up and configuring the container’s environment to run your application.

On a weekend I finally took out time for myself and sat down to embrace Docker, not only through the interactive tutorial on Docker’s website, but also on my server. I was half lucky in that I didn’t need to have to set Docker up on my local system or VM, because Linode just happened to very recently introduce support for Docker. I started playing around with Docker commands on the shell manually, and slowly transitioned to writing my own Dockerfile. The result: I wrote a Dockerfile to run a small container to run “irssi” inside it. Go ahead and check it out, please. If you have a system with Docker running on it (and I really think you should have one), you can follow the two or three commands listed on the README file to build and run my container. It is that easy!

Advertisements

Getting back inactive memory on Mac.

     OS X has the habit of keeping recently closed applications in memory so that if they are run again, they load quickly. The part of physical memory used for this purpose is called “Inactive memory”. The “System Memory” tab on the Activity Monitor application gives a break-down of the physical memory, including available free and inactive memory. Because of the way OS X behaves, you may or may not notice your system running low on “free” memory every now and then. This discovery could perplex you, because despite being low on free memory, you can load applications and go about doing your work. This is possible because inactive memory can be released by the OS X kernel’s memory management subsystem on demand. If it finds that the system is running short on free memory, and the user has started an application that is not already loaded in inactive memory, it will gladly comply and release enough of inactive memory to be able to run the requested application.

     I recently found a command line utility on OS X to release most of inactive memory. It is called, “purge”. The short description for “purge”, from its man page, states that its use forces the disk cache to be purged. The “disk cache” actually refers to “inactive memory”. To run this command, you have to type “purge” on Terminal.app (or any other Terminal application that you use). For example:

(Ayaz@mbp) [0] [~]
$ purge

     Before running the purge command, the memory breakdown on my system looked like:

     After the purge command ran, inactive memory went from 858MB down to 270MB.

     You will notice that the system becomes a little unresponsive while purge is flushing the disk cache. That’s fine and nothing to worry about.

     If you can’t find purge on your system, it could be because you have not installed XCode and accompanying development tools. These are available in one of the OS X installation discs. You can now also pay and download XCode from the Mac App Store.

     Have fun and be nice!

Converting a Git repo to Mercurial

     Until today, we had most of the projects on Mercurial repositories hosted on a Windows box in-house. The Mercurial web interface was set up to provide a convenient read-only access to both contributors and non-contributors within. However, the box in question was unreliable, and so was the Internet link it was alive on. And, in this world, there are few things more troublesome than having your source control server unavailable due to an unexpected downtime.

     What’s special about today is that we moved most of our Mercurial repositories to BitBucket.org. While they will remain private, the contributors as well as the internal non-contributors will be given access to them. Aside from having the repositories on a central server that we can mostly always rely on, having a lovely Web interface to browse and control the repositories, we also get useful goodies like issue trackers, wiki pages, and an easy user management interface. It is a win-win situation for everyone.

     One of the projects I was working on had only been on a local Git repository. I started work on it at a time when I found myself deeply in love with Git. Since BitBucket is a Mercurial repository warehouse, I had to find a way to convert or migrate the Git repository into a Mercurial one.

     I looked around on the Web and found a lot of people recommending the use of the HgGit plugin. As I understood it, this plugin made possible, among other things, the workflow that involved working on a Git repository and pushing the changesets to a Mercurial counterpart. However, the process of setting it up seemed rather complex to me. Plus, I didn’t want to keep the Git repository lying around when I was done with the migration. I wanted to be able to migrate the Git repository to a Mercurial one, push it upstream to BitBucket and make any changes in future in the source code by cloning from the Mercurial repository. What HgGit did seemed rather overkill for my needs.

     I then discovered the Mercurial ConvertExtension. This extension did just what I wanted: Convert repositories from a handful different SCMs into Mercurial. The process of converting a Git (or any other repository) to a Mercurial one through ConvertExtension is very straightforward.

     As a first step, you are required to edit your global .hgrc file to enable the extension thus:


[extensions]
hgext.convert=

     You are then required to run the hg convert command on your target Git repository thus:


$ hg convert /path/to/git/repo

     This will migrate the Git repository, and store it in the current directory inside a directory named repo-hg. Once inside the newly created Mercurial repository (and this part is important), you have to run the following command to checkout all the changesets:


$ hg checkout

     You may then push the repository with the usual hg push to BitBucket.

     :)

     PS: This blog post really helped get me going in the right direction.

Ranting about Mercurial (hg) (part 1)

     The day I got a grip of git, I fell in love with it. Yes, I am talking about the famous distributed version control application: git. I was a happy Subversion (svn) user before discovering git. The day to day workflow with git and the way git worked made me hate Subversion. I was totally head over heels in love with git.

     I have come to a point where the first thing I do when I am about to start a new project is to start a git repository for that project. Why? Because it is so easy and conveneient to do. I really like the git workflow and the little tools that come with it. I know that people who complain about git often lament about its lack of good, proper GUI front-ends. While the state of GUI front-ends for git has improved incredibly and drastically over the years, I consider myself a happy command-line user of git. All my interactions with git happen on the command-line. And, I will admit, it does not slow me down or hinder my progress or limit me in any way in what I do. I have managed fairly large projects on git from the command-line, and let me tell you, if you are not afraid of working on the command-line, it is extremely manageable. However, I understand that a lot of people run away from the command-line as if it were the plague. For those people, there are some really good git GUI front-ends available.

     I have to admit that for someone coming from no version control background or centralized version control background (such as Subversion or CVS), the learning curve to git could turn out to be a litle steep. I know that it was for me, especially when it came to getting to grips with concepts involving pushing, pulling, rebasing, merging, etc. I don’t think I will be off-base here to think that a good number of people who use version control software for fairly basic tasks are afraid of merge conflicts in code when collaborating code. I know I was always afraid of it, until I actually finally faced a merge conflict for the first time. That was the last time I felt afraid of it, because I was surprised to find out how easy it was to deal with and (possibly) resolve merge conflicts. There’s no rocket science to it.

     I have been plenty impressed with git that I wince when I am forced to use any other version control. In fact, I refuse to use Subversion, as an example, and try my best to convince people to move to git. Why? Because git is just that good. I should probably point the reader to this lovely article that puts down severals reasons that explain why git is better that a number of other popular version control applications out there.

     Recently, at work, I have had to deal with the beast they call Mercurial (hg). Yes, it is another famous distributed version control system out there. I was furious at the decision of my peers to select Mercurial for work-related use, and tried my best to convince them to use git instead, but they didn’t give in to my attempts to persaude them. I had no choice but to unwilling embrace the beast and work with it.

     Instantly, I had big gripes with the way Mercurial worked as well as its general workflow. Where I fell in love with how git tracked content and not files, I hated the fact that Mercurial only tracked files. What I mean by that is if you add a file in a Mercurial repository to be tracked and commit it, and then make any changes to the file, Mercurial would automatically add it to the staging commit. Yes, it wouldn’t let you specify manually which files that were changed should be part of the staging commit, something that git does. A staging commit is a staging area where files that have been changed and have been selected to be part of the next commit are loaded to be ready for commit. With git, you have to manually specify which file to add to the staging commit. And I loved doing it that way. With Mercurial, this was done automatically. This in turn brings me to the second big gripe I had with Mercurial: incremental commits. Git workflow places an indirect emphasis on incremental commits. What are incremental commits? In the version control world, it is considered best practice to perform a commit that is as small as possible and that does not break any existing functionality. So, if for example, you have made changes to ten files to add five different functionalities, that may be independent of each other, but would only like to commit one functionality at a time. That is where incremental commits come in action. With incremental commits, you specify only the files you want to be committed and commit them, instead of committing all the changed files in a bunch and add a general purpose commit message. Incremental commits is a big win for me, and I adore that feature. Because with git you had to manually specify which of the changed files you wanted in the staging commit area, incremental commits came easily and naturally. With Mercurial, where it automatically added all changes to tracked files into the staging commit area, it did not.

     So I asked the obvious question: How do we do incremental commits with Mercurial? Aftering sifting through the logs, I found out that it was indeed possible, but in a very ugly way. The Mercurial commit command takes an extra parameter, -I, that can be used to specify which files explicity to make part of the commit. But that wasn’t it. If you had, say, five files of the ten changed files you wanted to commit, you would have to tag the -I switch behind each of those files, otherwise, Mercurial will fail to include the files in the commit. To this day, this little quirk bites me every now and then.

     I will admit that the one thing I really liked about Mercurial was the built-in web server it came with that could provide a web-based interface to your Mercurial repositories for viewing. This was something that was missing from or quite difficult to do with git. And, this was also one of the biggest reasons why my peers at work decided to go with Mercurial for. They all wanted a convenient way to access the repositories over the web.

To be continued.

Jagged fonts on Snow Leopard on LCD over DVI

     I have a 13″ white MacBook. It has a glossy display that is about the best I have ever used in my life. Great quality, lovely fonts. The only drawback with it is a drawback with glossy displays: it reflects ambiant light which can at times cause problems. But I really don’t mind that. I can work around it easily. Overall, I am really happy with the display on the MacBook.

     The only problem is, 13″ is just not enough. Not all the time. But when you are managing many things side by side, and not least when writing code, you can really get frustrated from not having enough space on the screen. After all, it doesn’t help that you’ve windows hiding behind your current window in focus because there is no space left anywhere on the screen to toss two or three windows side-by-side and still be able to get anything done.

     So, a year ago, I purchased, after careful thought and research, a ViewSonic VX1940w external LCD. The reason I settled on this one, and not any other, is a combination of: a) great price I was getting on it; b) excellent resolution it was offering at that price; c) it sported both VGA and DVI inputs. I really, really wanted an LCD with a DVI input. If you haven’t clicked on the LCD catalog page, this LCD sports a max resolution of 1680×1050 and is 19″ in size and wide-screen. It has a matte display, in that, it does not have the reflection problem the glossy displays suffer from. But it isn’t nearly so crisp in display quality as the glossy.

     If you’ve had the occasion to use any Mac laptop, you’ll know that they don’t have the de facto VGA and DVI input/output ports. Instead they can have any one of mini-DVI, mini-VGA, mini-DisplayPort, or micro-DVI/VGA ports, which by the way are all Apple proprietary stuff. So, how do you connect almost all of the LCDs out there that come with standard VGA and DVI ports and connector wires? You buy VGA and/or DVI adpaters from Apple. These, I should mention, are anything but cheap.

     Naturally, I wanted to buy the DVI adapter for my MacBook. However, I did some research and found bad news in that, a lot of people on the Mac forums reported their LCDs not working with the DVI adapters Apple provides because of the latter being either DVI-I or DVI-D, and most LCDs not supporting one or the other. That scared me. The DVI adapter was expensive to being with, I was not sure whether it would really work with my LCD. So, I took the safe lane out and bought the VGA adapter. And it worked out of the box.

     But, there were problems. These were the sort of problems I documented in a question titled, “Why do I feel a strain on my eyes when I look at the 19″ matte display?“, that I asked on the Super User forum. You may want to skim through it to get the excruciating details of the problems I faced with the VGA display on the LCD. The bottom-line was that that I was told that I had to get the DVI adapter instead, and that because of the jagged fonts on the VGA display and generally sub-standard picture quality, my eyes were having a hard time adjusting, resulting in strain and headache. Initially, it didn’t make sense to me how that could really be the cause of the problem, and didn’t buy it. But, the fact remained that I couldn’t use the LCD for long without walking off with an incredible burden weighing down on my eyes.

     Finally, I bit the bullet today and bought a mini-DVI to DVI adapter. Having plugged it in, I noticed that the “auto image adjust” functionality on the LCD was disabled. It wasn’t when the VGA was plugged in. Despite changing a couple of settings that I thought might make a difference, I felt that the fonts looked even more jagged, the picture quality worse than before. Having spent some more time staring on the screen, opening up Terminals and editors and windows, I realized it wasn’t merely a feeling. The fonts and the display did look much worse than before. I freaked out. I didn’t know what to do. I rebooted the MacBook in the hopes that when I plug in the LCD again, I will actually surprise myself. To my utter dismay, nothing changed. I was sorely disappointed, and I didn’t know what to do. A feeling of remorse hit me for having spent a fortune on the DVI adapter only to get a crappier picture than I did on VGA.

     With a heavy heart, I started looking around. I skimmed with grief through a couple of similar posts on the forums about how fonts on DVI input on LCD looked jagged, and such. Nothing suggested really helped. And then, I found this blog post, Snow Leopard Font Smoothing. It talked about the exact same problem I was having. And what’s more, it suggested a fix for the problem. The fix was to toggle a global setting on OS X by running the following command on the Terminal.app:

defaults -currentHost write -globalDomain AppleFontSmoothing -int 2

     I did this, but nothing worked. I wasn’t really sure what else I had to do to get this to work, neither did the blog post hinted at anything beyond running that command. After a while, I was convinced that there was no hope for me in this. I had nearly given up when I noticed the fonts on the text on the menu bar. It looked different. It was crisper, better, more beautiful. And then it hit me. I quickly, first of all, quit my Terminal.app, and when I re-opened it, voila! There they were. The lovely fonts I had fallen in love with on the MacBook screen. I restarted one by one most of the applications. Apparently, what little helpful bit that blog post missed out on was that you had to close all your applications and re-run them for the settings to take effect. I imagine an easier way is to simply log off and log back in.

     I am absolutely ecstatic with all this. Not only can I use my big LCD to manage my work in a better and more productive manner, I don’t have to walk away with unbearable headaches and eye-strain. Well, I still have headache and eye-strain, to be honest, but that’s an every-day thing. I am so happy.

Using mouse inside Vim on Terminal.app

     When I am writing code, I spend most of my time inside Vim on Terminal.app on MacOS. When I am not writing code, I still end up spending a good bit of my time on Terminal.app, running all sorts of commands, using command-line applications (such as irssi for IRC), and editing files in Vim here and there.

     Coming from Linux, my biggest gripe with Vim on Terminal.app has been the fact that I could not scroll through the Vim window using the wheel on my mouse as I would any normal editor. If I tried to scroll up or down, Terminal.app would scroll through the Terminal session instead and mess everything up. For a long time, I had to contend myself with holding down the Fn and Shift keys and using the Up and Down arrows to scroll back and forth. Anyone who has done this can immediately imagine how annoying this can get.

     So, the other day I had a thought cross my mind and I asked myself, “Vim can’t really be that lame and not support the functionality of scrolling with the mouse properly. I am simply missing out on something.” I looked around, and sure enough, I found that Vim does support all sorts of mouse-related operations on the shell, including properly scrolling up and down. This particular feature of Vim is toggled via the “:set mouse=a” setting. You could go through the help menu, with “:help mouse” inside a Vim session, to see what other values, apart from “a”, that setting can take. But for all intents and purposes, the “a” setting is sufficient.

     I was happy Vim could do that. But, when I toggled it in Vim on Terminal.app, it didn’t work. Scrolling, as before, would only scroll through the Terminal.app buffer and mess up things. I was disappointed.

     Sifting through a couple more threads here and there on the web about Vim, scroll, and Terminal.app, I was able to figure out the reason why, after enabling mouse support on Vim, it didn’t work on Terminal.app. As it turns out, Terminal.app eats away all the mouse gestures thrown at it, and does not delegate them to Vim or other applications running within the shell. It doesn’t let them through. Vim wasn’t getting any mouse gestures at all. This was a problem. I thought I had bumped into a big, thick wall that could not be crossed.

     But, I was wrong. Luckily, I somehow managed to find a plugin, by the name of Mouse Term, for Terminal.app that patched Terminal.app’s behaviour of masking mouse gestures and not letting them through to individual applications. Mouse Term, as is mentioned on its home page, requires another, small application, SIMBL, to be installed first to work. In fact, Mouse Term is a SIMBL plugin.

     Once I had SIMBL and then Mouse Term installed and Terminal.app loaded, I could actually scroll through Vim, with the “:set mouse=a” setting, prefectly. Not only that, I noticed something else as well that made me so happy. I use tabs in Vim. Vim makes available half a dozen Vim commands, that start with the prefex “:tab”, to manipulate tabs, navigate through them, etc. If you’ve ever used tabs seriously on Vim, and used these commands to cycle through tabs, for example, you’d understand how frustrating it can get. But, guess what my excitement was about? Not only could I now scroll through a Vim window smoothly, I could also click on the tabs to switch to them. I could also click on the “X” on the top right corner to close a tab. It simply worked.

     The best part is when you have the NERD Tree Vim plugin installed. You can, inside the NERD Tree window, expand and collapse directories and open files simply at the click or double click of the mouse. As a Vim user, what more could you want?

Pakistan Summer Time, and NTP on OS X

     I noticed today that, after a major update to OS X along with a security update, the time on the system clock was an hour ahead. In fact, I didn’t pick it up until after I had glanced at the time on my cell phone. When I opened the preferences where different settings related to time and date can be set, I realised that the Network Time Protocol (NTP) had been enabled, which meant that the system was syncing time and date, along with the usual time zone information, from a remote network time server. In my case, that server was time.asia.apple.com, one of three servers in the drop-down list of NTP servers in the preferences to choose from.

     As with the other two, time.asia.apple.com is an NTP server that is managed by Apple themselves. If you travel a lot, or if you are mindful of and in a place where daylight savings time is commonplace, being able to use an NTP server to not worry about having to change time and date is ideal. It is convenient. After all, time is important, and keeping track of time more so.

     Now, I love NTP. It sure beats having to change time manually all the time. But, what if the NTP server you so dearly depend on suddenly starts spewing out incorrect time? Well, you’d eventually notice that, yes, but it would be annoying. The emails you send are suddenly ahead of time, the IM messages you receive as well, your calendar events, etc. If the difference in time due to the error is subtle, say, maybe off by an hour or so, you will likely take longer to spot it. Not that your house will burn down, or your business will plummet in a downward spiral into loss, but it sure will cause problems, even if little, annoying ones.

     So, why am I here on a hot Saturday afternoon with no mains power, talking about all this? Because I found out today that time.asia.apple.com is giving out a time for Pakistan that is +6 GMT, when it should correctly be +5 GMT. Judging from the label “Pakistan Summer Time” that the NTP server is using to describe the time, I can understand where this skew in time is creeping in from. But it is wrong. And the time on my system is wrong. What’s worse is that the place in system preferences where date and time settings are, does not provide an option for me to use a custom NTP server of my own choosing. I am restricted to choosing from the drop-down of three NTP servers, only one of which applies to my time zone. Bugger!

     Until I found /etc/ntp.conf This small text file stores the address of an NTP server to use. Regardless of whether you have NTP time enabled in the preferences pane, you will have an existing entry in the file. If you change the address in there to point to something, say, asia.pool.ntp.org, the system will use the new NTP server. In the preferences, the NTP server you added will automatically be selected for you, even though, if you pull the drop-down, you won’t notice it in the choices available.

     The only problem is, asia.pool.ntp.org also has Pakistan time pinned down at +6 GMT. Square one!