FFXIV 1.0 Classic Server Dev Blog

What's Happened Since Last Post????

2017-03-10T20:07:24+00:00

Hoookay, I think it's about time that I did another blog post. To those who thought the project was dead, fear not! It's been chugging along on/off. While I haven't posted any blog posts since then, I have posted a few videos of various features that were completed. One of the people helping me; Jorge, had kept a list of thing I've completed since then... omg this will be a lot to go through!

Overhaul of the Actor and Script Systems; Doors, Npcs Galore!

When I first started the project, the server could only spawn one instance of a "class" of actor. This meant only one Goobbue, one crab, one door, etc. This would not do as obviously there would be multiple copies of these actor classes. The table got split into two. The gamedata_actor_class table which defines what script to load for this class id, the attributes about the class, and the trigger events. More specific things like the position and idle animation were moved to a spawn_location table.

Following this addition, it opened up a huuuge amount of possibilities. Most of Limsa, Gridania, and Ul'dah is populated (if not all). All doors work in those cities work as well. This allows for free movement of the city w.o having to resort to warp.

Scripts Part 2

Later I rewrote how the system handles calling client functions. You used to have to keep track of where the client is since the last call and have these huge if statements. Now it's as simple as:

choice = callClientFunction(player, "delegateEvent", player, man0l0Quest, "processEvent020_9");

The client runs the "processEvent020_9" function in it's internal scripts, and sends the response back as a "choice" variable. I have just finished overhauling the lua engine (again), and will soon be able to support "waiting" as well, for X seconds or a signal for even more advance scripts.

Tons of Opcodes

I probably have 90% of the opcodes in the game figured out. Things from the "pushEvent triggers" which setup an event to fire when you "touch" something, to the generic data packet, to even the full cutscene book packet, which means no more crashing on the "Path Companion" cutscenes!

Rework of the Server Architecture

Back then all communication between client and server was done through one program. This would not do for a multi-server architecture which FFXIV was. I have split the server into a "World" server which controls all global things (zoning, linkshells, friendlists, etc), and a "Zone" server which controls all game stuff for each zone (battle, events, updates).

BG Objects

Airships, ferries, and doors oh my! Another set of opcodes figured out was the bg object opcodes. BG Objects are... objects saved in the client's map file that can be told to animate or show/hide by the server. These are things like above mentioned, but also things like doors, clams (from that dungeon I am forgetting to spell), etc. I've even figured out how to find other ids in the map to control even more objects!

Coming Soon...

Currently on the dev branch are some other features waiting to finishing completion or bugtesting. Groups are done, which covers parties, linkshells, and soon; content groups. Directors have also been completed allowing for finer control of content. Private Areas; instanced sub areas of a zone, are also mostly done. With those two features, I was able to build the entire opening event for Limsa sans the battle itself. We also dabbled a bit in getting read for a battle system.... successfully extracting the map files into an OBJ format. I'll be posting more, but subscribe to my Youtube channel as well, as I post videos a bit more often. There are tons of smaller features I haven't written about, but hopefully this satiates some of your appetite. Until next time!

Shout Outs

Just a thank you to Devi, Demo, Jorge, Mordred and all those in Discord for helping and supporting along the way.

Just a reminder, source code is available on Bitbucket

2016-06-06T15:22:10+00:00

Since a lot of people are asking and may have missed it (I don't think I ever posted outside of Reddit posts), here is the Bitbucket Git repo:

https://bitbucket.org/Ioncannon/ffxiv-classic-server

also a wiki me and some other people helping with the project created:

http://ffxivclassic.fragmenterworks.com/wiki/index.php/Main_Page

Opening scenes half-scripted

2016-04-23T12:47:47+00:00

I've been working hard on figuring out how the opening scenes of 1.0 worked. It's been tough because a lot of the packets I have were recorded back pre-Yoshi (or sometime a bit after), way before the tutorials in 1.2x got introduced. Still after trial and error and some research into what was happening in other captures... I was able to get all three opening scenes working up to the battle portion of the game. Here are two recordings I did, though note the Limsa one was a lot earlier:

As you can see with the Gridania one, it stops once the battle starts as I have been having trouble with those tutorial scripts. I decided to take a break from the opening scene and working on the "group" packets which deal with.... well groups. Originally this was actually the hardest packet to figure out when Localhost was still stuck in the inn. After seeing how the items and gamemessage packets work, it was easy to see that a lot of them were just 8/16/32/64 sized versions of the same thing. I got most of them figured out and have begun writing out the code. These "groups" are use for player parties, monster parties (not sure what the means yet.... linking?), linkshells, duty content (when you see that "You are bound to duty", this means a group was created), and relation groups which deal with invites. Here are some screenshots of my experimentation with relation groups:

One other thing I never mentioned back before the work on opening scenes started was class switching is basically done. I've added code allowing the user to store gearsets as well as switch their weapon. When you equip an item, the equipment is saved into the db at a per-class basis. So, switching your weapon causes the server to load up the last set you had for that class. It then searches through your inventory for the correct items and equips them. The exception is undergarments/shirts which stay the same across the board.

Equipment almost completed, played around with the original launcher/updater

2016-03-08T17:57:21+00:00

I'm almost done building the equipment system. Equipping items, unequipping items, and updating the player graphic is completed. All that is left is to implement examining other players. The whole item list had to be uploaded into the server database to do checks like if a item has been max-stacked or what type of item it is. Sadly graphical data is not stored on the client (unlike 2.0) and requires me inputting every model id. Luckily these match pretty well with 2.0 as the whole db was imported when they transitioned. Here are some tests I did:

I also took a look at the original login and updater exes to see if I could get them working, instead of using SU (for that authentic effect). First I had to figure out the updater's version check request. This turned out easy at first and became very difficult. This updater and it's server is the same one used to update 2.0's launcher (the version check/download that happens in the grey box before the launcher appears). Using Wireshark and Fiddler I saw the HTTP GET request the program makes. The API works like this:

A request is made to "<website>/vercheck/ffxiv/win32/release/game(or boot)/<patchnumber>". If the version is up to date, the server responds with

header("HTTP/1.0 204 No Content");
header("Content-Location: ffxiv/2d2a390f/vercheck.dat");
header("X-Repository: ffxiv/win32/release/boot");
header("X-Patch-Module: ZiPatch");
header("X-Protocol: torrent");
header("X-Info-Url: http://www.example.com");
header("X-Latest-Version: 2012.09.19.0000");

If there is an update, it responds with

header("HTTP/1.0 200 OK");
header("Content-Location: ffxiv/48eca647/vercheck.dat");
header("Content-Type: multipart/mixed; boundary=477D80B1_38BC_41d4_8B48_5273ADB89CAC");
header("X-Repository: ffxiv/win32/release/boot");
header("X-Patch-Module: ZiPatch");
header("X-Protocol: torrent");
header("X-Info-Url: http://example.com");
header("X-Latest-Version: 2012.05.20.0000.0001");
header("Connection: keep-alive")

--477D80B1_38BC_41d4_8B48_5273ADB89CAC
Content-Type: application/octet-stream
Content-Location: ffxiv/48eca647/metainfo/D2012.05.20.0000.0001.torrent
X-Patch-Length: 20874726
X-Signature: jqxmt9WQH1aXptNju6CmCdztFdaKbyOAVjdGw_DJvRiBJhnQL6UlDUcqxg2DeiIKhVzkjUm3hFXOVUFjygxCoPUmCwnbCaryNqVk_oTk_aZE4HGWNOEcAdBwf0Gb2SzwAtk69zs_5dLAtZ0mPpMuxWJiaNSvWjEmQ925BFwd7Vk=

[TORRENT FILE HERE]
--477D80B1_38BC_41d4_8B48_5273ADB89CAC--

These are both HTTP responses. They are technically copy/pastes of some PHP code on my server. First; in the header is the patch format (ZiPatch), the protocol to download (can by HTTP or Torrent), latest version info, and some general HTTP stuff. If there is a patch, a body entity is also added, with the content location, patch length, a signature, and then either the HTTP file or Torrent file.

About the content-location; both define a directory in your downloads folder under /My Documents/My Games/Final Fantasy XIV/downloads. The header version is to store the vercheck.dat file... this is basically the response saved into a file. The one in the entity body is where to download the http or torrent file to.

In 1.0, all patches were signed and the "X-Signature" gives the signature to verify the patch with. Strangely this was no longer done in 2.0. After these headers, the torrent or http file is appended. All of the patch stuff in 1.0 was done through torrents. However in 2.0 the patches are downloaded using http, and a simple file is sent, consisting of a bunch space-seperated-values including the url to download the patch at.

After a bunch of debugging and trial and error I was able to get the updater to work. To by-pass the signature issue, I found the "RSA_verify" function (part of the OpenSSL library) in ffxivboot.exe and forced it to always return 1 (return true; it's verified). Sadly there seems to be some limitations with the torrent client. I think the client will only talk to certain trackers or peers, and explains why the official updater was so slow compared to normal torrent clients. I shelved work on the updater till a later time.

Regardless, even if I couldn't patch, I could at least inform the updater it was up to date and start the login client. It's a very simply program that shows a Internet Explorer Frame. If a downloaded HTML page has this tag:

<x-sqexauth sid="$sessionId" lang="en-us" region="2" utc="11231131" />

It will start ffxivgame.exe with the given arguments. Now to get the official login client to work... I had to finish one of the last missing core components of the lobby server: the blowfish key generation. All of the lobby traffic is encrypted using Blowfish encryption. The client grabs the current computer time at launch and send that to the server unencrypted. Along with a phrase "Test Ticket Data" to help generate the key. It combines these two pieces of info and some other constant numbers, it then MD5 hashes this into a 128bit key for Blowfish.

With Seventh Umbral (and my server as well until now), the SU launcher was forcing the time to a constant value. On the server side it used the same key at all times. With the original login client, it would send a new time value to the server at each launch so key generation had to be completed. Well, now it is and you can login using either clients!

I got a friend who is way better at webdev than me to write a look-a-like of the original login page: