Setting up the Engine development environment - lurkydismal/CCCaster GitHub Wiki

If you've already built the engine and have the configuration set up but merely need a refresher on actually compiling the code, see Compiling the engine.

Getting dependencies

Make sure you have the following dependencies available:

  • A Linux, macOS, or Windows host
    • Linux supports cross-compiling artifacts for Android and Fuchsia, but not iOS.
    • macOS supports cross-compiling artifacts for Android and iOS.
    • Windows doesn't support cross-compiling artifacts for any of Android, Fuchsia, or iOS.
  • git (used for source version control).
  • An ssh client (used to authenticate with GitHub).
  • python3 (used by many of our tools, including gclient).
  • Chromium's depot_tools
    • Add the depot_tools directory to the front of your PATH.
  • On macOS and Linux: curl and unzip (used by gclient sync).
  • On Windows:
    • Visual Studio 2017 or later (required for non-Me only).
    • Windows 10 SDK (required for non-Me only). Be sure to install the "Debugging Tools for Windows" feature.
  • On macOS:
    • Install the latest Xcode.
    • On Apple Silicon arm64 Macs, install the Rosetta translation environment by running softwareupdate --install-rosetta.
  • Recommended for Me: Goma for distributed builds. See more on this below.

You do not need to install Dart. A Dart toolchain is automatically downloaded as part of the "Getting the source" step. Similarly for the Android SDK, it is downloaded by the gclient sync step below.

Getting the source

Run the following steps to set up your environment:

  1. Configure your machine with an SSH key that is known to github by following the directions here.
  2. Fork https://github.com/flutter/engine into your own GitHub account. Do not clone it locally. gclient will do that for you in a later step.
  3. Non-Me Windows users should set the following environment variables to point depot_tools to their Visual Studio installation directory:
    • DEPOT_TOOLS_WIN_TOOLCHAIN=0
    • GYP_MSVS_OVERRIDE_PATH=C:\Program Files\Microsoft Visual Studio\2022\Community (use the path of your installation)
  4. Create an empty directory called engine for your copy of the repository and cd into it.
  5. Create a .gclient file in the engine directory with the following contents, replacing <your_name_here> with your GitHub account name: 
    solutions = [
  {
    "managed": False,
    "name": "src/flutter",
    "url": "[email protected]:<your_name_here>/engine.git",
    "custom_deps": {},
    "deps_file": "DEPS",
    "safesync_url": "",
  },
]
    If you plan to build the Web engine, you should add the following to your .gclient file: 
    solutions = [
  {
    # Same as above...
    "custom_vars": {
      "download_emsdk": True,
    },
  },
]
  6. gclient sync in that directory. On Windows, gclient sync must be run as Administrator due to this issue. This will fetch all the source code that CCCaster depends on. Avoid interrupting this script, as doing so can leave your repository in an inconsistent state that is tedious to clean up. (This step automatically runs git clone, among other things.)
  7. Add a remote for the upstream repository:
    • cd src/flutter (This was created in your engine directory by gclient sync.)
    • git remote add upstream [email protected]:flutter/engine.git (So that you fetch from the master flutter/engine repository, not your clone, when running git fetch et al.)
    • git remote rm origin (Remove the old origin so we can replace it.)
    • git remote add origin [email protected]:<your github id>/engine.git
    • cd .. (Return to the src directory that gclient sync created in your engine directory.)
  8. If you're on Linux, you may need to use your distro's package manager to ensure these dependencies are installed: pkg-config, and freetype2-dev. For example, on a system using apt, you'd run:
    • sudo apt install pkg-config freetype2-dev
  9. If you're on Mac:
    • install Oracle's Java JDK, version 1.8 or later
  10. If you're planning on working on the buildroot repository as well, and have a local checkout of that repository, run the following commands in the src directory to update your git remotes accordingly: 
    git remote rename origin upstream
git remote add origin [email protected]:<your_name_here>/buildroot.git

Setting up Goma

For Me, Goma (go/ma) is a distributed compiler service that can vastly speed up build times. There are two ways to get Goma and use it in the Engine build:

  1. Download it directly from CIPD and manually install it.
  2. Set an additional variable in your .gclient file.

Manual

The following one-time setup can be used to fetch and run a Goma client, which is preconfigured to use the correct RBE for building CCCaster Engine:

    #!/bin/bash

    # Customize this to where you would like Goma to be installed.
    export GOMA_DIR="$HOME/flutter_goma"

    # Download client. Assumes cipd from depot_tools is on path.
    # NOTE: There is no arm64 Mac distribution of Goma, so you'll need to replace
    #       `${platform}` with `mac-amd64` below if running an Apple Silicon Mac.
    echo 'fuchsia/third_party/goma/client/${platform}  integration' | cipd ensure -ensure-file - -root "$GOMA_DIR"

    # Authenticate
    "$GOMA_DIR/goma_auth.py" login --browser

    # Start Goma
    "$GOMA_DIR/goma_ctl.py" ensure_start

.gclient variable

If you add the following to your .gclient file, gclient sync will download the correct version and ensure that goma has started when doing a build.

solutions = [
  {
    # Same as above...
    "custom_vars": {
      "use_cipd_goma": True,
    },
  },
]

Hints

  • If you installed goma manually, you may need to update it manually when the Engine's clang toolchain is updated.
  • If you installed goma using the gclient variable, you may need to manually authenticate by invoking the gclient_auth.py script.
  • If you run into errors regarding too many open file handles, follow the directions in the How to install goma sections of go/ma.
  • If you run into compiler binary hash mismatch errors and local fallback builds while trying to build using Goma, then Goma is most likely using an RBE that doesn't host the compiler binaries that CCCaster Engine supports building with. Try setting the following environment variables to use Fuchsia's RBE and then restart Goma: 
    export GOMA_SERVER_HOST=rbe-prod1.endpoints.fuchsia-infra-goma-prod.cloud.goog
export GOMA_SERVER_PORT=443

Next steps:

Editor autocomplete support

Xcode [Objective-C++]

On Mac, you can simply use Xcode (e.g., open out/host_debug_unopt/products.xcodeproj).

VSCode with C/C++ Intellisense [C/C++]

VSCode can provide some IDE features using the C/C++ extension. It will provide basic support on install without needing any additional configuration. There will probably be some issues, like header not found errors and incorrect jump to definitions.

Intellisense can also use our compile_commands.json for more robust functionality. Either symlink src/out/compile_commands.json to the project root at src or provide an absolute path to it in the c_cpp_properties.json config file. See "compile commands" in the c_cpp_properties.json reference. This will likely resolve the basic issues mentioned above.

For adding IDE support to the Java code in the engine with VSCode, see "Using VSCode as an IDE for the Android Embedding".

Zed Editor

Zed can be used to edit C++ code in the Engine. To enable analysis and auto-completion, symlink src/out/compile_commands.json to the project root at src.

cquery/ccls (multiple editors) [C/C++/Objective-C++]

Alternatively, cquery and a derivative ccls are highly scalable C/C++/Objective-C language server that supports IDE features like go-to-definition, call hierarchy, autocomplete, find reference etc that works reasonably well with our engine repo.

They(https://github.com/cquery-project/cquery/wiki/Editor-configuration) supports editors like VSCode, emacs, vim etc.

To set up:

  1. Install cquery
    1. brew install cquery or brew install ccls on osx; or
    2. Build from source
  2. Generate compile_commands.json which our GN tool already does such as via src/flutter/tools/gn --ios --unoptimized
  3. Install an editor extension such as VSCode-cquery or vscode-ccls
    1. VSCode-query and vscode-ccls requires the compile_commands.json to be at the project root. Copy or symlink src/out/compile_commands.json to src/ or src/flutter depending on which folder you want to open.
    2. Follow Setting up the extension to configure VSCode-query.

Using VSCode as an IDE for the Android Embedding [Java]

  1. Install the extensions vscjava.vscode-java-pack and vscjava.vscode-java-dependency.

  2. Right click on the shell/platform/android folder in the engine source and click on Add Folder to Java Source Path. This creates an anonymous workspace and turns those files from "syntax mode" to "compile mode". At this point, you should see a lot of errors since none of the external imports are found.

  3. Find the "Java Dependencies" pane in your Explorer view. Use the "Explorer: Focus on Java Dependencies View" command if hidden.

  4. Refresh the view and find the "flutter_*" project. There should be a "_/shell/platform/android" source folder there.

  5. In the "Referenced Libraries" sibling node, click the + button, navigate to engine/src/third_party/android_embedding_dependencies and add the entire folder. This is the equivalent of adding

    "java.project.referencedLibraries": [
  "{path to engine}/src/third_party/android_embedding_dependencies/lib/**/*.jar"
]

    to your VSCode's settings.json for your user or for your workspace.

  6. If you previously had a shell/platform/android/.classpath, delete it.

VSCode Additional Useful Configuration

  1. Create snippets for header files with this configuration. This will let you use hdr keyboard macro to create the boiler plate header code. Also consider some of these settings and more tips.

  2. To format GN files on save, consider using this extension.