A compare and contrast of game post mortems - Falmouth-Games-Academy/comp350-research-journal GitHub Wiki
What is a post mortem?
A post mortem can have very various shapes, but it’s, most of the time, a simple text document. Its main purpose is to keep track of the development of a game 1(http://lvictorino.com/blog/write-post-mortems.html). A post mortem should assist you in developing better products within the future and hopefully enable the team to work more efficiently. Most postmortems will only be accessible for the members of the development team, although post-mortems will occasionally be released to the public eye. They can be a great insight to those interested in game development to know what not to do when developing their own games. Post-mortems should be compressed into direct issues rather than have mass amounts of un-needed info on a certain component, a post mortem should follow a strict quality over quantity layout.
What to put in a post mortem
Post-mortems should contain hard facts about the game rather than only highlight the greatly accepted components of your game, even if your game has exceeded the expectations of your expected sales figures and target audience. Every game will have its flaws whether they be minor or major, it is essential to make note of them within your post mortem to avoid them in future endeavours. A good post mortem should cover development conditions, techniques, game design direction, code architecture… the game in its entirety 1(http://lvictorino.com/blog/write-post-mortems.html). Asking the correct questions about the development of your game will ensure you will gain an accurate post mortem that will prove to be useful in future. Rather than focus on the guilty party for any of the flawed components, you should focus on how to correct the mistakes to ensure they won't be performed again. You won't be able to find the perfect solution/s to every noted flaw, but try to find leads on how to improve your development scheme and habits 1(http://lvictorino.com/blog/write-post-mortems.html).
When should a post mortem be written?
Post-mortems are usually written a few months after the release of the game to get accurate feedback about what players both liked and disliked about the game. It is also a good idea to start writing a post mortem within each stage of the development stages of the game, this way you can prevent making previous mistakes later on in the development and therefore produce a better product for the target audience. Making notes on critical bugs and balancing issues (if your game contains a competitive multiplayer aspect) for the game throughout the development will make life much easier in future when avoiding creating the same or a similar bug in future development.
A comparison of post-mortems
GamaSutra has a good number of heartfelt post mortem written by the developers themselves 2(http://www.gamasutra.com/view/news/238773/10_seminal_game_postmortems_every_developer_should_read.php). While the majority are honest and transparent, some can read proud and boastful. Despite this, it is possible to infer development or technical optimisation decisions that have been made. It is also apparent that many of the games referenced below are relatively old (by the fast paced nature of the games industry), but the sentiments and approaches to development are relevant regardless of the software version or RAM requirements. The reasoning of professional developers when evaluating optimisation under industry pressures is an invaluable insight into lessons learned from experience. Both good and bad.
Everybody overscopes
With many games striving to exceed the current hardware standards, visual prowess is a common starting point. However, new and advanced technologies are often poorly documented and unoptimised. Sometimes with a long-planned development cycle, the target hardware is not yet a consumer standard. Black & White required 64MB of RAM at a time where 32MB was commonplace 3(http://www.gamasutra.com/view/feature/131476/postmortem_lionhead_studios_.php?page=2). Developers can choose to wait for the standard to rise, or more often will be forced to change their design. Deus Ex developers originally planned to build large, outdoor areas mimicking real sections of New York City 4(http://www.gamasutra.com/view/feature/131523/postmortem_ion_storms_deus_ex.php?page=3), but this was unfeasible given the standard of renderers available then. Thief developers chose to write a custom renderer but suffered when the developer who wrote the renderer left the company 5(http://www.gamasutra.com/view/feature/131762/postmortem_thief_the_dark_project.php?page=2). Any further optimisations were extremely risky to carry out. Maintainability and reliability versus using cutting edge or optimised software seems to be a prevalent choice for many developers.
Third party middleware
However, using third-party solutions can also be risky. Rainbow Six developers chose to use licensed rendering software and networking technology 6(http://www.gamasutra.com/view/feature/131829/postmortem_redstorms_rainbow_six.php). Only to find that the required interface code between the licensed Java code and their own C++ code to be larger than the entire networking library! The renderer was also packaged as an unalterable black box, so they had to stop using both mid-development. The synergy between all software aspects significantly speeds up development time. Consider that the tools used to create Diablo II did not operate within the game engine, so changes to lighting and animations had to suffer a full build in order to be viewed 7(http://www.gamasutra.com/view/feature/131533/postmortem_blizzards_diablo_ii.php?page=3). Not being able to use Profiling techniques will undoubtably slow down Measuring Performance significantly. Deus Ex found that, although using Unreal Engine may have saved considerable engine development time, they did not understand the code as well as would have had they created it themselves 4(http://www.gamasutra.com/view/feature/131523/postmortem_ion_storms_deus_ex.php?page=3). Ultimately this led to them shipping the game with Direct3D performance issues. A tricky problem to fix without changing Unreal Engine itself.
Time to work faster
Common across many post-mortems is the theme of needing more time. Baldurs Gate II developers sensibly invoked asset size restrictions on audio, animation, and characters per area 8(http://www.gamasutra.com/view/feature/131493/baldurs_gate_ii_the_anatomy_of_a_.php)). However, a relatively inexperienced development team of designers and artists would sometimes break these restrictions. leading to frantic optimisation late in development. Similarly, Myth: The Fallen Lords 9(http://www.gamasutra.com/view/feature/131691/postmortem_bungies_myth_the_.php) struggled with large textures and animation files. These were cached to help memory usage, but again blame running out of time as to why they could not ultimately fix the problem. Interestingly, with small teams and indie ventures where a set release date is not a factor, the opposite may be true. Armadillo Run 10(http://www.gamasutra.com/view/feature/2728/indie_postmortem_armadillo_run.php) acknowledges that premature optimisation can spoil workflow at crucial stages, instead highlighting the need to write correct optimised code as a standard at the time.
Regarding time constraints, sometimes the most sensible idea with solving a problem is to simply drop the feature entirely. Similar to Deus Ex and their plan for large outside areas 4(http://www.gamasutra.com/view/feature/131523/postmortem_ion_storms_deus_ex.php?page=3), Myth developers chose to completely remove 3D particle effects that were found to slow the framerate 9(http://www.gamasutra.com/view/feature/131691/postmortem_bungies_myth_the_.php), and Unreal Tournament dropped from 16 256*256 textures per player to 3 or 4 for memory consideration 11(http://www.gamasutra.com/view/feature/131569/postmortem_epic_games_unreal_.php). Optimisations in these post-mortems include any measure taken to 'do less work', not specifically within the code. Design choices such as System Shock 2 choosing to reuse technology from Thief: The Dark Project 5(http://www.gamasutra.com/view/feature/131762/postmortem_thief_the_dark_project.php?page=2), also include optimisation in terms of reusing assets 4(http://www.gamasutra.com/view/feature/131813/postmortem_irrational_games_.php). An understandable design choice if there are a limited number of designers 12(http://www.gamasutra.com/view/feature/131813/postmortem_irrational_games_.php). Interestingly because of this, Thief and System Shock 2 (two very different games) used the same executable and simply chose a different object hierarchy and data set at run time.
Cutting edge vs usability
Ultimately, with the new technology that games developers are required to use to stay current, documentation and tools can be sparse. Despite being established, System Shock 2 developers found that The Dark Engine lacked many tools needed for debugging and analysis 12(http://www.gamasutra.com/view/feature/131813/postmortem_irrational_games_.php). Even the Unreal Engine veterans that developed Unreal Tournament chose to use a modular code structure for ease of use. With platform specific code in separate libraries, and each subsystem connected through clearly defined interface 11(http://www.gamasutra.com/view/feature/131569/postmortem_epic_games_unreal_.php). Their requirement for ease of use and maintainability exceeded the value of a few frames that may have been saved by optimising over an increased development time.
The choice to be made between licensing existing technology or developing the system themselves is made by balancing time constraints, and the potential for a more optimised system. System Shock 2 developers chose to alter their design to the technology, rather than building technology to fit the design 11(http://www.gamasutra.com/view/feature/131813/postmortem_irrational_games_.php). Ultimately, sharing an adaptable system with the Thief developers was revealed to be advantageous as there grew a support network between the two development teams. The Dark Engine uses an easy to use, data-driven approach involving an intuitive file structure and hierarchy that does not require recompilation when changing assets. This also allows off the shelf tools for constructing and viewing resources 5(http://www.gamasutra.com/view/feature/131762/postmortem_thief_the_dark_project.php?page=2). The developers were able to restrict the potential overscoring on run time expense that an open source engine may leave possible, by using custom built C++ pseudo scripts to limit access into the code 5(http://www.gamasutra.com/view/feature/131762/postmortem_thief_the_dark_project.php?page=2). It is worth noting that this was before Quake and Unreal Engine popularised this approach of exposing unharmful variables to designers.
Tools help
With larger games requiring larger teams, the potential for unoptimised systems increases. The Witcher 3 developers handled this in a particularly interesting way. By converting all level data that potentially needs to be optimised (such as LODs that need fixing) into a database, they then use SQL commands to query for potential problems. An example of a real command that they have used is:
SELECT DISTINCT depotpath, triangleCount, author, autoHideDistance FROM MeshComponents WHERE lodCount < '1' AND triangleCount >=2000 AND autoHideDistance >=50
or
SELECT posx, posy radius FROM Lights WHERE shadowCasting = false
When optimising and debugging the developers can get vertex counts and bounding box areas of mesh components to quickly discover assets or areas that need attention. Their database viewer could then open the editor at the appropriate asset chosen to greatly speed up the optimising process, fighting the time constraint clause blamed by many other post-mortems. This can then be exported to Google Docs, for lists to give to developers detailing optimisation tasks. This example shows how a correctly implemented tool can greatly speed up optimisation and debugging. In addition to this case, vertex density maps that are designed to show areas that need to be optimised also helped show sparse areas of the map that needed to be further populated 13(https://www.youtube.com/watch?v=p8CMYD_5gE8).
Developers used this tool to find a chicken bucket with 1,500,000 triangles and a shadow casting light that was 4,000 metres in diameter!
Another good example of good tools aiding development was The Chinese Room's Amnesia: A Machine for Pigs 14(http://www.gamasutra.com/view/feature/218372/postmortem_the_chinese_rooms_.php). During the development of this game the HPL2 engine 15(https://wiki.frictionalgames.com/hpl2/start) was easily modifiable by TCR's programmer to create plugins and change the engine to help the team work to their full potential14(http://www.gamasutra.com/view/feature/218372/postmortem_the_chinese_rooms_.php).
Communication is Key
One of the primary successes for The Chinese Room's Amnesia: A Machine for Pigs was the solid inter team communication. The core development team's ability to work well together with minimal man-management required was one of the most obvious successes of the development, and without doubt influenced the quality and cohesiveness of the final product. Multiple members of the team had previously worked together on past projects such as the Half-Life 2 mod Off Limits, which meant there was an established rapport between them. It was then very simple for the more junior staff to find their own suitable fit within the team14(http://www.gamasutra.com/view/feature/218372/postmortem_the_chinese_rooms_.php).
Porting to iOS
When first making The Witness they already encountered some memory footprint issues with their lightmaps and that was when the game was being made for Computers 15(http://www.ludicon.com/castano/blog/2016/09/lightmap-compression-in-the-witness/). So when they decided to port it to a platform requires an even smaller memory footprint they had to look for further ways to optimise their lightmap technology 15(http://www.ludicon.com/castano/blog/2017/10/lightmap-optimizations-ios/). With a game whose gameplay highly relies on how things look the team had to be extremely careful with the entire porting process 16(https://www.gamespot.com/articles/the-witness-dev-on-overhauling-the-game-for-mobile/1100-6453873/). For the final iOS release the developers were able to reduce the lightmap memory footprint by 83% which the achieved by doing least squares vertex baking 17(https://www.cs.utah.edu/~ladislav/kavan11least/kavan11least.pdf).
References
[1] http://lvictorino.com/blog/write-post-mortems.html
[2] Gamasutra, 10 seminal game postmortems every developer should read, accessed 5th February 2019.
[3] Gamsutra, Postmortem: Lionhead Studios' Black & White, accessed 11th Febuary 2019.
[4] Gamasutra, Postmortem: Ion Storm's Deus Ex, accessed 11th February 2019.
[5] Gamasutra, Postmortem: Thief: The Dark Project, accessed 11th February 2019.
[6] Gamasutra, Postmortem: Redstorm's Rainbow Six, accessed 11th February 2019.
[7] Gamasutra, Postmortem: Blizzard's Diablo II, accessed 11th February 2019.
[8] Gamasutra, Baldur's Gate II: The Anatomy of a Sequel, accessed 11th February 2019.
[9] Gamasutra, Postmortem: Bungie's Myth: The Fallen Lords, accessed 11th February 2019.
[10] Gamasutra, Indie Postmortem: Armadillo Run, accessed 11th Febuary 2019.
[11] Gamasutra, Postmortem: Epic Games' Unreal Tournament, accessed 11th February 2019.
[12] Gamasutra, Postmortem: Irrational Games' System Shock 2, accessed 11th February 2019.
[13] GDC, The Witcher 3: Optimizing Content Pipelines for Open-World Games, accessed 11th February 2019.
[14] Gamasutra, Postmortem: The Chinese Room's Amnesia: A Machine for Pigs, accessed 12th February 2019
[15] HLP2 Engine Documentation