FastFiles and Zone files (AW): Difference between revisions

From COD Engine Research
Red-EyeX32 (talk | contribs)
Created page with "Category:FastFiles Category:AW == FastFile Structure == enum compress_type_t { zlib = 1, lzx = 2 }; //Entry point for the FF. struct DB_Header { ..."
 
Red-EyeX32 (talk | contribs)
 
(15 intermediate revisions by 3 users not shown)
Line 3: Line 3:


== FastFile Structure ==
== FastFile Structure ==
  enum compress_type_t
<source lang="cpp">
   {
enum DB_CompressorType
    zlib = 1,
{
    lzx = 2
   DB_COMPRESSOR_INVALID = 0xFFFFFFFF,
   };
  DB_COMPRESSOR_ZLIB = 0x1,
  DB_COMPRESSOR_LZX = 0x2,
   DB_COMPRESSOR_PASSTHROUGH = 0x3,
};
   
   
//Entry point for the FF.
struct GfxImageStreamData
struct DB_Header
{
{
  int unknown[3];
  char magic[8]; // S1ff0100
};
  int version;
 
  bool allowOnlineUpdate;
struct FF_Header
  compress_type_t compressionType;
{
  char unknown1;
  char magic[8]; // S1ff0100
  char unknown2;
  int version;
  int unknown3;
  bool compress;
  int unknown4;
  DB_CompressorType compressType;
  int padding;
  char sizeOfPointer;
  int fileSize;
  char sizeOfLong;
  int maxFileSize;
  int fileTimeHigh;
};
  int fileTimeLow;
  int imageCount;
  GfxImageStreamData imageData[imageCount];
  int baseFileLen;
  int totalFileLen;
};
</source>


=== Magic ===
=== Magic ===
Line 29: Line 38:
=== Version ===
=== Version ===
Helps to dictate the version of the fastfile, so Ghosts fastfiles cannot be loaded on Advanced Warfare. The Advanced Warfare version is 0x72E for the Playstation 3 and Xbox 360 consoles, and the same for PC.
Helps to dictate the version of the fastfile, so Ghosts fastfiles cannot be loaded on Advanced Warfare. The Advanced Warfare version is 0x72E for the Playstation 3 and Xbox 360 consoles, and the same for PC.
<source lang="cpp">
  if (version > 0x72E)
  if (version > 0x72E)
   Com_Error("Fastfile for zone '%s' is newer than client executable (version %d, expecting %d)", version, 0x72E);
   Com_Error(ERR_DROP, "Fastfile for zone '%s' is newer than client executable (version %d, expecting %d)", filename, version, 0x72E);
  else if (version < 0x72E)
  else if (version < 0x72E)
   Com_Error("Fastfile for zone '%s' is out of date (version %d, expecting %d)", version, 0x72E);
   Com_Error(ERR_DROP, "Fastfile for zone '%s' is out of date (version %d, expecting %d)", filename, version, 0x72E);
=== AllowOnlineUpdate ===
</source>
Whether or not assets in this fastfile may be replaced by a fastfile loaded at a later time.
=== Compress ===
=== Entries ===
Whether or not the zone is compressed inside the fastfile.
These are still not entirely understood. Most fastfile systems simply parse past them.
=== Image Data ===
==== Playstation 3 & Xbox 360 ====
These are still not entirely understood.
ReadBytes(entryCount * 0x0C);
=== File Sizes ===
=== File Sizes ===
The fileSize is the size of the fastfile, everything included. The maxFileSize is the highest number that fileSize can be, and in most fastfiles they are equal.
The baseFileLen is the size of the fastfile, everything included. The totalFileLen is the highest number that baseFileLen can be, and in most fastfiles they are equal.
== Sub Header ==
== Sub Header ==
The rest of the fastfile is read depending on what the main fastfile magic was. If the magic was "S1ff0100" then the fastfile is signed. If it was "S1ffu100", the fastfile is unsigned.
The rest of the fastfile is read depending on what the main fastfile magic was. If the magic was "S1ff0100" then the fastfile is signed. If it was "S1ffu100", the fastfile is unsigned.
Line 46: Line 55:
The only data remaining is compressed data, depending on the enum in the header of the fastfile. Decompressing the data yields the [[#Zone File|zone file]].
The only data remaining is compressed data, depending on the enum in the header of the fastfile. Decompressing the data yields the [[#Zone File|zone file]].
=== Signed Files ===
=== Signed Files ===
<source lang="cpp">
  struct DB_AuthHash
  struct DB_AuthHash
  {
  {
Line 51: Line 61:
  };
  };
   
   
  struct DB_AuthSignatureHash
  struct DB_AuthSignature
  {
  {
   char bytes[256];
   char bytes[256];
};
struct DB_AuthSubheader
{
  char fastfileName[32];
  int reserved;
  DB_AuthHash masterBlockHashes[244];
  };
  };
   
   
Line 68: Line 71:
   int reserved;
   int reserved;
   DB_AuthHash subheaderHash;
   DB_AuthHash subheaderHash;
   DB_AuthSignatureHash signedSubheaderHash;
   DB_AuthSignature signedSubheaderHash;
   DB_AuthSubHeader subheader;
   struct
  {
    char fastfileName[32];
    int reserved;
    DB_AuthHash masterBlockHashes[244];
  } subheader;
   char signatureBlock[0x1E030];
   char signatureBlock[0x1E030];
  };
  };
</source>
The signatures on the XBlocks are checked as the fastfile loads. Then the sub-header master block hashes are checked, followed by the signedSubheaderHash, and lastly the subheaderHash. If any fail, then the loading of the FF is aborted. Only Activision and the game developers can generate the RSA signatures.
The signatures on the XBlocks are checked as the fastfile loads. Then the sub-header master block hashes are checked, followed by the signedSubheaderHash, and lastly the subheaderHash. If any fail, then the loading of the FF is aborted. Only Activision and the game developers can generate the RSA signatures.
==== Magic ====
==== Magic ====
The subheader magic is always "S1ffS100".
The subheader magic is always "S1ffS100".
=== PS3 ===
=== PS3 ===
After the zone file is created, it is split into blocks each 0x10000 bytes. Each block is compressed using default zlib compression. The size of the compressed block is stored as an unsigned 16 bit integer over the zlib header (0x78DA). Compressed blocks are concatenated and appended to the fastfile.
After the zone file is created, it is split into blocks each 0x10000 bytes. Each block is compressed using default zlib compression. The size of the compressed block is stored as an unsigned 16 bit integer over the zlib header (0x78DA). Compressed blocks are concatenated and appended to the fastfile.
== Zone File ==
== Zone File ==
The zone file is the decompressed data from an FF.
The zone file is the decompressed data from an FF.
=== Header ===
<syntaxhighlight lang="cpp">
#define XFILE_BLOCK_TEMP 0x0
#define XFILE_BLOCK_PHYSICAL 0x1
#define XFILE_BLOCK_RUNTIME 0x2
#define XFILE_BLOCK_VIRTUAL 0x3
#define XFILE_BLOCK_LARGE 0x4
#define XFILE_BLOCK_CALLBACK 0x5
#define XFILE_BLOCK_SCRIPT 0x6
#define MAX_XFILE_COUNT 0x7
#ifdef PS3
  #define MAX_XFILE_COUNT 0x8
#endif
struct XFile
{
  unsigned int size;
  unsigned int externalSize;
  unsigned int blockSize[MAX_XFILE_COUNT];
};
</syntaxhighlight>
=== Asset List ===
<syntaxhighlight lang="cpp">
struct XAsset
{
  XAssetType type;
  XAssetHeader header;
};
struct ScriptStringList
{
  int count;
  const char **strings;
};
struct XAssetList
{
  ScriptStringList stringList;
  int assetCount;
  XAsset *assets;
  XGlobals *globals;
};
</syntaxhighlight>

Latest revision as of 04:40, 31 March 2015


FastFile Structure

enum DB_CompressorType
{
  DB_COMPRESSOR_INVALID = 0xFFFFFFFF,
  DB_COMPRESSOR_ZLIB = 0x1,
  DB_COMPRESSOR_LZX = 0x2,
  DB_COMPRESSOR_PASSTHROUGH = 0x3,
};
 
struct GfxImageStreamData
{
  int unknown[3];
};

struct FF_Header
{
  char magic[8]; // S1ff0100
  int version;
  bool compress;
  DB_CompressorType compressType;
  char sizeOfPointer;
  char sizeOfLong;
  int fileTimeHigh;
  int fileTimeLow;
  int imageCount;
  GfxImageStreamData imageData[imageCount];
  int baseFileLen;
  int totalFileLen;
};

Magic

The magic is a string that is 8 characters long. It is either "S1ff0100" or "S1ffu100". It indicates what is read after maxFileSize. If it is "S1ff0100" then this fastfile is signed. If it is "S1ffu100", then this fastfile is unsigned.

Version

Helps to dictate the version of the fastfile, so Ghosts fastfiles cannot be loaded on Advanced Warfare. The Advanced Warfare version is 0x72E for the Playstation 3 and Xbox 360 consoles, and the same for PC.

 if (version > 0x72E)
  Com_Error(ERR_DROP, "Fastfile for zone '%s' is newer than client executable (version %d, expecting %d)", filename, version, 0x72E);
 else if (version < 0x72E)
  Com_Error(ERR_DROP, "Fastfile for zone '%s' is out of date (version %d, expecting %d)", filename, version, 0x72E);

Compress

Whether or not the zone is compressed inside the fastfile.

Image Data

These are still not entirely understood.

File Sizes

The baseFileLen is the size of the fastfile, everything included. The totalFileLen is the highest number that baseFileLen can be, and in most fastfiles they are equal.

Sub Header

The rest of the fastfile is read depending on what the main fastfile magic was. If the magic was "S1ff0100" then the fastfile is signed. If it was "S1ffu100", the fastfile is unsigned.

Unsigned Files

The only data remaining is compressed data, depending on the enum in the header of the fastfile. Decompressing the data yields the zone file.

Signed Files

 struct DB_AuthHash
 {
   char bytes[32];
 };
 
 struct DB_AuthSignature
 {
   char bytes[256];
 };
 
 struct DB_AuthHeader
 {
   char magic[8]; // S1ffS100
   int reserved;
   DB_AuthHash subheaderHash;
   DB_AuthSignature signedSubheaderHash;
   struct 
   { 
     char fastfileName[32];
     int reserved;
     DB_AuthHash masterBlockHashes[244];
   } subheader;
   char signatureBlock[0x1E030];
 };

The signatures on the XBlocks are checked as the fastfile loads. Then the sub-header master block hashes are checked, followed by the signedSubheaderHash, and lastly the subheaderHash. If any fail, then the loading of the FF is aborted. Only Activision and the game developers can generate the RSA signatures.

Magic

The subheader magic is always "S1ffS100".

PS3

After the zone file is created, it is split into blocks each 0x10000 bytes. Each block is compressed using default zlib compression. The size of the compressed block is stored as an unsigned 16 bit integer over the zlib header (0x78DA). Compressed blocks are concatenated and appended to the fastfile.

Zone File

The zone file is the decompressed data from an FF.

Header

#define XFILE_BLOCK_TEMP 0x0
#define XFILE_BLOCK_PHYSICAL 0x1
#define XFILE_BLOCK_RUNTIME 0x2
#define XFILE_BLOCK_VIRTUAL 0x3
#define XFILE_BLOCK_LARGE 0x4
#define XFILE_BLOCK_CALLBACK 0x5
#define XFILE_BLOCK_SCRIPT 0x6
#define MAX_XFILE_COUNT 0x7
#ifdef PS3
  #define MAX_XFILE_COUNT 0x8
#endif


struct XFile
{
  unsigned int size;
  unsigned int externalSize;
  unsigned int blockSize[MAX_XFILE_COUNT];
};

Asset List

struct XAsset
{
  XAssetType type;
  XAssetHeader header;
};

struct ScriptStringList
{
  int count;
  const char **strings;
};

struct XAssetList
{
  ScriptStringList stringList;
  int assetCount;
  XAsset *assets;
  XGlobals *globals;
};