This project has to be my largest and most far reaching one yet. Put simply, the Mimesis concept is a tool for distributing, on a large scale, modded content for titles on the Xbox console. It was written in C++/DirectX to be compiled as an xbe (Xbox Executable) to be ran on a modified Xbox console.
Installation of modded content on an Xbox for titles such as Halo 2 was cumbersome, required knowledge of the system/title in order to get it working, and was limited to only those with modified Xbox’s who could play from a backup of their game on the harddrive (since you cannot edit files on the title’s dvd disc). In the case of Halo 2, there were tutorials showing how to add a modded map to the title’s map select list, but this was just not enough. I wanted a much easier and intuative process. Just as I did with the HMPi project, I wanted an automatic installer for the content, only this time it would run on the Xbox itself. It would talk to a content server in order to get the latest list of available content then allow the user to select which he would like and it would install it automatically. It would work even when playing from the original game disc obviously without modifying any files on that disc.
Xbox Live! Content
Starting 2 weeks after the title was released, myself and a few others online began research in how Halo 2 would likely format its downloadable content (DLC). It wouldn’t be until April of the following year that we would get our first glimpse at any real content when Bungie released an AutoUpdate which allowed for Halo 2 to download content from the Xbox Live! service onto the Xbox harddrive that worked while running from the original title’s disc! This was exactly what I was hoping to accomplish with modded content and my own server. So I started to imagine how we could trick the Halo 2 game into connecting toour server instead of the Xbox Live! servers hence seeing our modded content and downloading it instead. How we were to package the content so that Halo 2 would play it was still an unknown at this point, we simply wanted to figure out if we could spoof the Xbox into connecting to our servers. We tossed around ideas of a DNS spoofer, eventually tossing it asside in order to pursue an even more novel concept; create our own format and distribution method and write our own Xbox program to interface it. All it would require is figuring out how the content is packaged to allow Halo 2 to play it off the harddrive while running from the disc.
Through the research phase fo the project I had developed quite the following in the HXBdev.org forums (since closed down). One of those who worked so diligently with me on the concept, and just seemed to get it, was Ramzi. We were both on the same page about the project and were we wanted to take it. So we started work on Project Mimesis (then called HDLC or Homebrew Downloadable Content). Having never written any C++ or DirectX (which was required in order to develop an application on the Xbox), both of us tried pitching the concept to known developers in the community, to no avail. We weren’t met with the warmest of welcomes, mostly getting ridiculed for it being a dumb idea or something similar. Those who didn’t ridicule the idea were either to busy to help or didn’t know anymore than Ramzi and I in regards to writing an Xbox program. We grew discouraged and let the idea sit until we could find someone who was able to write it.
Time to learn C++ and DirectX
A couple months went by and it was becoming painfully obvious that no one was going to help us write this program. If we wanted it to happen, we were going to have to do it ourselves. Ramzi and I started reading everything we could find about Xbox development. We took examples from sites dedicated to it and tried to get them to work, analyzing each line of code along the way. We learned how to do it on our own and began writing the program that we had been ridiculed for over the past two months, knowing full well that this was not a dumb idea but a great one, one that many users would enjoy, including ourselves.
Over the next couple of months Ramzi and I worked feverously on the code learning so much along the way. I was writing all of the server side PHP code which would be responsible for serving/listing the content to the Xbox application via http/xml. It was in it’s infancy but was already starting to show promiss that we may actually be able to do what we set out to do; write a program which can download content from a central server and install it onto the Xbox without user interaction.
Just a little reverse engineering
It was May of 2005 when Ramzi and I started to make some real headway in our efforts to figure out the downloadable content format for Halo 2, announcing on May 4th that the patch_v3.lvl format (a key security file for a Halo 2 content package) had been completely mapped out. It was not long after that announcement when a fellow modder by the name of CLuis contacted me and offered his help in the reverse engineering process. At this point we had one major hurdle left and it was the digital signature which was written into the header of the lvl files. Ramzi and I had yet to figure it out at this point when CLuis came in and cracked it about 2 weeks later. It was a great day for the project since that digital signature was the last unknown preventing us from accomplishing everything we wanted to. Now the only hurdle was our lack of C++ abilities, but that wouldn’t stop us for long.
Finally some respect
It’s June at this point and we finally launched the domain which was dedicated to the project: projectmimesis.com. With a forum section and regular news posts on the front page updating the public about the progress of the project’s development, Mimesis was really starting to take off. The news posts were getting regular attention from the largest Xbox news site, xbox-scene.com which only fueled the public interest. People were starting to get it, they were starting to understand why I was so adamant about what I had envisioned so long ago.
Not only were the community of eager users supporting us more than ever, but we began to talk with other development teams responsible for Xbox applications such as XBMC, Avalaunch, & YSBox. JbOnE and Team Avalaunch really went out of their way to help us by offering code taken from their dashboard in order to get Mimesis moving forward. They also took extra time to work with me directly on basic C++ related questions I had, being that this was the first time I had ever written in the language. We really owe them a great deal for their willingness to help out; Mimesis is only where it is because of the great people that make up the Xbox modding community like them, so with the help of Team Avalaunch, we finally had support for compressed content packages, a real necessity due to the large files that made up a downloadable content package such as a new multiplayer map for Halo 2. With files as large as 40MB at the time, bandwidth was deffinately going to become an issue.
Version 1.0 Released
Click here to see a video of version 1
Development up to the release of version 1.0 was spotty at best, mixed in with a kitchen remodel as well as major problems with the computers in the house (I got hit with a nasty worm which took a while to clean out). So it was late August before the first version of Mimesis saw the light of day as a publicly released application. It was a very exciting day for Ramzi, CLuis, and myself, because it was the dream being made into a reality as we watched the server logs with such great interest while users, one by one, ran the program, connected and downloaded content. It was quite a wonderful sight.
A Distributed Load
Due to the growing popularity of the Mimesis concept and the size of the mods which were to be hosted on the service I got to work right away on a solution. From early estimates, I figured I was looking at a potential bandwidth usage of 332 gigabytes per month! This was going to be hard to pull off without costing me an arm and a leg in usage fees from the hosting company. A dramaticly different approach was going to be needed in order to allow the project to grow into the service I had dreamed of, so I began brainstorming. I came up with the idea to create a vast system of mirrors which could be submitted by the users themselves and distributted evenly when users are downloading content, so that it doesn’t always have to come dirrectly from our servers. A user could download a master copy of the content package from the mirror submission page, host it on their own webserver, then submit the url to the mirror system using the form provided on the website. With a very robust and automatic mirror authentication system in the backend, I was able to allow this system to work on its own, nearly unattended. When a user posted a new mirror url, the server verified it automatically by doing a server to server transfer then by calculating an MD5 hash of the package after it had been downloaded. This also allowed for the measuring of the speed in which the mirror performed, which was used for ranking and ultimately, along with other information gathered about the mirror url, used in the load balancer by the content server when the Mimesis app (running on a user’s Xbox) requested a download. It also had self checking mechanisms in place which would police the quality of the mirrors disabling any which became broken or too slow.
Mimesis Featured on the G4 Gaming Channel
As if seeing the application used by fellow Xbox modders wasn’t enough, I learned less than one month after our release that Mimesis was being featured on the popular video game channel, G4 (formerly TechTV)! I couldn’t believe it until I actually saw it with my own eyes. Overtaken by the excitement of being recognized by such a prestigious gaming channel as G4, I litterally screamed as I sat next to my girlfriend watching the show. I mean I watched TechTV all the time, I never in a million years would have thought that something I did would have been featured on it! Video of the program we had been writting over the last few months, my interface design, our code was on TV being played back for millions to see. It was an absolutely sureal experience, and I thought to myself, the only thing that could have made this better was if it was the dark tipper who actually presented it, but thats really old school. You can watch a video clip of it here.
Mimesis version 2
Fueled by our recently successful version 1 release we quickly got started on version 2. We had a lot of ideas on how we could improve the application and take it from a proof of concept to a polished bug free application with more features than ever before. Introduced in version 2 was the ability to download content for games other than just Halo 2. Complete with a redesigned interface and features that went above and beyond expectations such as AutoUpdate management and multigame support, we released version 2 on November 21st, 2005 to much fanfair.
Taking a break
After the release of version 2, Ramzi, CLuis and I wanted to take a break so we did. We didn’t work on the next version of Mimesis for the next couple of months, instead spending more time managing the content servers and enjoying the second version. Around Febuary is when the guys that managed the XLink Kai servers setup a Mimesis only room for players to play Mimesis content against each other over the Kai tunneling service. This was a pretty big deal since no other modding group had made such an impact on the community as Mimesis seemed to be doing. Shortly after, in March 2006, XBConnect (the other tunneling service) also created a Mimesis specific section in thier system. At this point the website was getting about 2.5 million hits a month and the content was being delivered by the mirror system without a hitch.
It wasn’t long before Ramzi and I started to get the itch to rewrite Mimesis. We had learned so much along the way and coded ourselves into a corner in the process. So we werew really eager to take our new knowledge of C++ and DirectX and try to apply it in the form of a completely re-written version of Mimesis from the ground up. Ramzi started in on what was to be a feature filled, fully robust rendering engine that we could use to enable skins for Mimesis. I helped where I could but he really had a handle on things and cranked that code out pretty fast. I worked on rewritting how the content was installed and managed adding lots of new features along the way including a completely new way to manage the existing content on your Xbox, allowing the user to enable/disable anything with the click of a button. Another major improvement was the development of a brand new interface and the ability to use the dvd remote to control Mimesis.
After lots of late nights and countless hours put into programming version 3, Ramzi and I were able to release it on July 3rd, for the holiday. It was our greatest release yet, with tons and tons of people flocking to the site to see the news and grab the latest content. The stats for Mimesis usage shot through the roof while we saw 5.5 Million hits on the site that month. Mimesis had been launched 140 thousand times by over 20 thousand unique users. It was a successful release to say the least. With over 100 content packages available for download and growing, Mimesis is used daily by gamers out there who want to get the latest and greatest in modded content with the easiest install process, allowing for a non modded Xbox to play content with a modded Xbox. The idea is a reality now and I am pretty proud of how it turned out.