Colin Godsey

I Hate Sockets… and HTTP

Colin Godsey — Sat, 06 Mar 2010 00:08:35 +0000

TCP/IP programming has been a really ‘fun’ experience so far in my journey to create a JavaScript web server. Starting with sockets all the way to HTTP, I have succeeded in destroying my confidence in my ability to understand computers. In case any of you suffer from this same affliction dealing with sockets, here is a little tip that may help you get to bug #341: Content-Length Means Bytes, Not Characters.

Unique Object Delayed Write w/ Redis

Colin Godsey — Thu, 03 Dec 2009 01:37:57 +0000

Trying to keep your data updated to a database while undergoing heavy traffic can lead to a very slow response for your queries. Ever wish you could push your writes to a queue so they can be handled during system time and not user time? It’s pretty easy. This model requires there to be a non-disk key/value data store (Redis, memcache) available that can act as a forward cache. This works better if the system can support lists. If an object does not exist in the forward cache when asked for, it needs to be pulled from the database, assembled and stored in the forward cache. When an object is updated, it needs to be updated in the forward cache as well as being pushed to the write queue. This way your user script only accesses the forward cache and the write cache during standard calls when the cache is already populated with the needed data.

This trick can be enhanced even further by storing a list of unique IDs in the write cache that can be replaced when an objected is updated before one of it’s previous writes was processed. This means that what would have been two database writes is now one! This requires storing a list in cache as well as the objects, which is done very effectively with Redis.

When used correctly and with a properly designed database that reduces the number of unique objects, your writes are faster and fewer! With a few more tricks this system can also be used easily and effectively in a cluster environment. More on that later.

Redis: Persistent Memory Key-Value Store

Colin Godsey — Sat, 21 Nov 2009 03:02:22 +0000

Just today I was challenged with a project that will require value caching in a PHP/MySQL cluster environment. This project will have to deal with loads of up to 500 requests per second, most of which will involve writes. When working on a small scale it is easy to forget the latency issues associated with all engines in MySQL. This project will require a massive read/write caching system that will sit between PHP and MySQL and will require full data persistence. I found a valuable article that shows some of the timing differences of the popular storage systems under load. Redis seems to be the real winner here. With write-only data persistence (write to disk), this storage engine still comes out on top! This is truly the best system if you are caching write calls to MySQL. This way, most of the data will survive in case of a crash. Persistence can be increased even further by using another fully redundant Redis server. This is a must have for high-load write caching.

Logos Engine – HDR Video

Colin Godsey — Thu, 29 Oct 2009 20:56:45 +0000

This video showcases some new features of the deferred rendering system in my Logos Engine including HDR, auto-exposure, edge detect anti-aliasing, shadow mapping and more. Check the page for more info.

Click here to view the embedded video.

Deferred Lighting and Shadow Mapping

Colin Godsey — Wed, 07 Oct 2009 07:09:29 +0000

I just finished implementing my deferred lighting and shadow mapping system for the T2 (Logos) engine that will be used for my upcoming structure based game. Here is a nice little screenshot of the whole deferred lighting system with HDR, depth of field and shadow mapping using OpenGL 2.1 post-processing techniques. The tank is just there to show off the shadows.

Jungle Disk with Amazon S3 – Holy Grail of Offsite Data Storage

Colin Godsey — Mon, 05 Oct 2009 22:33:54 +0000

Using Jungle Disk with Amazon S3 data storage I have finally become completely data secure. On top of that, I pay a price better (and more appropriately adjusted) price than I could ever have hoped for.

Generally when files sit on my computer, I may have one copy of them. Sometimes I will have a hard drive elsewhere where I keep copies of those files, because as we all know hard drives come and go. This system allows me to keep my data at any point in time (sometimes two) with the ability to lose one of those ‘points’ at any time. I have used different version control systems before and it gives me the great ability to control my data at a variety of fixed points in time per set of files. So I can keep a version control system and make automated back-ups of that. This isn’t terribly practical for data like images, videos of important things in your life, source code, private writings, important songs, the things you don’t ever want to lose, and things that don’t change much. It would be nice to take all of these concepts and just boil it down, right? I think I have found that Holy Grail of data storage.

Using Jungle Disk with Amazon S3 has given me the most awesome combo of personal data backup ever. With my current set-up I have a standard Amazon Simple Storage Service (S3) Account which attaches to my standard Amazon account I use for ordering books. This gives me access to my “cloud data”.

Amazon distributes this data out all over the web and it is failure proof. The website states:

Failure tolerant: The system considers the failure of components to be a normal mode of operation, and continues operation with no or minimal interruption.

That is a powerful statement, but with cloud technology and RAID storage this service is one of the most powerful data storage system for every purpose. It is marketed towards web developers for use in websites, cloud computing, and high intensity global distribution, but it’s even really awesome just for your laptop backups and it’s dirt cheap!

They price based on a varied scale. It breaks down into a price you have to pay per GB of storage per month and a price you have to pay per GB of storage to access it (read or write). At a glance, the numbers don’t make sense and you can’t even begin to imagine how much it will cost you to use it for backups. But I decided to go for it. Using a piece of software called Jungle Disk I was able to interface with this data as an interesting type of backup media. Jungle Disk allows incremental backups (only changed data) and also offers version control. This way when you don’t make much change to your data, it costs you way less to keep it up to date using Jungle Disk and you can keep a time range of data also. Last month (Sept. ‘09) I paid the following for my Amazon S3 account:

$0.150 per GB – first 50 TB / month of storage used	22.243 GB-Mo	3.34
$0.100 per GB – all data transfer in	2.558 GB	0.26
$0.170 per GB – first 10 TB / month data transfer out	1.073 GB	0.18
$0.01 per 1,000 PUT, COPY, POST, or LIST requests	48,355 Requests	0.48
$0.01 per 10,000 GET and all other requests	206,830 Requests	0.21
		4.47

What that really says is that I maintained 22.2 GB of important data (including 6 days worth of file version history for modified and deleted files of all types), 2.6 GB of transfered data that came out of my computer, plus 1.1 GB of data that came out of their cloud, I only spent $4.47. This is also for 2 computers, both of which I use for continuous work. If I didn’t make that many changes to my local set of data it would have only been $3.34 for that month. That’s more than I spend at Taco Bell after a night at the bars. I would much rather have my data safe.

The Jungle Disk software reliably and cheaply synchronizes your local and cloud data over the span of however many days you told it to. Jungle Disk itself only costs $2.00 per month. This is amazing! Not only is my data reliably stored at two places (one of which is indestructible) but it’s protected from me by giving me a set of data over a time frame that I can continue to access. You can even set it to never delete a file if you want. Jungle Disk will also hook you up with a network drive that you can treat basically like a hard drive. This data stays separate from your backup data, but it is charged just the same and is stored in the same cloud “bucket”.

This has been the best backup solution I have seen so far for my personal data and my personal business. Even for enterprise data it’s perfect (all of these transactions can be encrypted of course). Back-ups of this site as well as my other sites also sit on there.

Draw, Josh.

Colin Godsey — Mon, 17 Aug 2009 16:20:09 +0000

My buddy Josh Peter has been on his Daily Stranger escapade for 54 days already. This adventure includes drawing a sketch of a stranger (daily) and publishing it on his blog draw, josh. These drawings have kept my Google Reader account lively and entertaining for almost two months now! I recommend you check it too. Here’s a fav:

Daily Stranger 36

Eclipse and Zend PDT Icons for Ubuntu/Linux

Colin Godsey — Wed, 05 Aug 2009 21:52:28 +0000

I use Eclipse for all sorts of stuff including Zend PDT, Android SDK and C++. I like to keep my desktop looking well organized and looking professional. I never liked the default Eclipse icon that came with the Linux installs, so I made my own for Eclipse and Zend PDT icons with transparency that look snazzy with blending in GNOME panels or Avant Window Manager.

Coming Soon… Structure & T2-Engine

Colin Godsey — Mon, 13 Jul 2009 03:52:08 +0000

Hey guys, keep tuned for updates on CRG Studios’ upcoming simulator: Structure. It is past concept and will soon be impacting your world. The engine this will be using will be my open source T2 Engine. This game will be cross-platform, hardware enabled and terribly awesome.

Click here to view the embedded video.

Amarok 2 Under Windows

Colin Godsey — Mon, 27 Apr 2009 18:59:50 +0000

I know, I know. Some would say it’s gross or perverse, but in reality it is undeniably awesome. You can install Amarok 2 under Windows using The KDE on Windows Project. I’ve always been a fan of Amarok and have found Winamp and other various Windows based music players just don’t do it as well as Amarok. The only difficulty I had installed this was finding the native Amarok package in the KDE4W installer. I had to pick version 2.2.1 instead of the head stable release to find a prebuilt and working package for Amarok but alas I have it working!