Settings Improvement Design Document - bdring/Grbl_Esp32 GitHub Wiki

Status: Implementation available PR #408 implements a new settings mechanism based on the design goals and ideas given below. Here is a brief overview of the new scheme:

• The "classic Grbl" syntax like "$100=20.0" is supported, for compatibility with existing senders and user knowledge.
• Each setting has a text name like "xStepsPerMM", in addition to the old name like "100". When new settings are added in the future, there will be a text name, but probably not an old-style numeric name. The numeric names will exist only for legacy settings.
• WebUI setting names like "ESP108" are similarly supported, and they also have text names like "ApChannel"
• Any setting can be set using either the $key=value syntax or the [key]value syntax, where the "key" can either be a number, and ESPnnn name, or a text name. In the past, the $ syntax was used only for numbers and the [] syntax only for ESPnnn names. Now either syntax can be used with any form of key name.
• Settings name entry is case-insensitive, so it does not matter whether you write "xStepsPerMm" or "sstepspermm" or "ESP108" or "esp108".
• The existing "$$" command for listing settings values in numeric form is still supported, but it only works for old grbl-compatible numbered settings. There is a new "$S" command that lists all settings in text form.
• You can display the current value of any setting with, for example, "$100" or $xstepsPermm" or "[esp100]" or "[yacceleration]".
• Settings are stored in non-volatile storage in a "key=value" form, so that firmware updates, including ones that change the list of settings, do not invalidate the settings store.
• At init time, settings are read from non-volatile storage and cached in RAM, so subsequent access is fast.

• Changes to the settings structure cause loss of user-configuration / reversion to defaults.
• Numbered settings are hard for users to remember (but on the positive side, are compatible with Grbl Classic).
• The current method for inventing new settings that are not present in Grbl Classic is somewhat ad-hoc - just pick some hitherto-unused numbers. That doesn't extend well.
• The current settings are sort of grouped in 10s. Limits/Homing are in the $20s, Spindle stuff is in the $30s. This is very limiting and breaks down if the count of a group goes over 10.

• Settings should persist across firmware updates. If new firmware adds additional settings, the old settings should still be honored.
• If you downgrade to old firmware, any new settings that are not understood by the older firmware should be ignored so the old firmware does not break
• It should be possible to revert all settings to the default values and start over
• Setting names should be easy for humans to remember (not arbitrary numbers)
• The setting name scheme should be designed with future evolution in mind
• It should be possible to make a settings-configurable firmware build that can handle a substantial variety of machines. "One firmware handles every possible machine" is probably too much of a stretch, but it should be possible to cover a pretty big space in one build. This would avoid the need for everybody to do compiles.
• The complete list of settings should be easy to pass around in text form, to help with support. Like they way you can currently capture the output of $$, but even easier - and including the info in the startup message, so you don't have to capture that as a separate step.
• Settings should be processor-agnostic as far as possible.
• Settings should accommodate different machine classes - mills, lasers, lathes, plasma cutters, 3d printers, ...

• To guard against breakage when upgrading/downgrading, don't change the meaning of an existing setting. Value sets can be extended in a compatible fashion with care. Be very careful about changing the basic way of specifying something. Case in point - g2core changed the set of names for enabling spindles; the new name set was "better", but it caused a lot of sender breakage. If you need to invent a new scheme, continue to honor the old scheme, and do not reuse any old names in the new scheme with different meanings; create new names.
• It is probably better to store the settings in the main FLASH - perhaps in SPIFFS - rather than on an SD card.
• There should be some way to validate the integrity of the settings store, but it should not be so strict/rigid as to violate the upgrade/downgrade goals.

Currently some preferences are stored in the .ini file format. See preferences. Here are some current stats.

• WiFi Settings
  • UsedEntries:223
  • FreeEntries:407
  • All Entries:630
• used = 7136B
• free = 13024B
• 13K available

There is an ESP32 Preferences library that could be suitable for implementing the settings store. You use it like this:

#include <Preferences.h>

Preferences preferences;
preferences.begin("grbl");  // "grbl" is the "namespace"
preferences.getFloat("SpindleMaxSValue", 100.0);  // Second arg is default in case the store doesn't have that key
preferences.getUInt("XAxisStepPin", -1);

• Axes
  • Step pin
  • direction pin
  • limit switch pin
  • CS pin (SPI)
  • Ganged
  • Squared
  • Steps per mm
  • Max Velocity
  • Acceleration
  • Max Travel
  • microsteps
  • Run current
  • hold current
  • stallguard value
• Homing/Limit Switches
• Motion control
  • Enable pin
  • Step Idle Delay
  • RMT
  • Junction Deviation
  • Arc tolerance
  • Enable Trinamic (might be inferred from other settings)
• Reporting
  • Status report options
  • Units
• Spindle
  • Spindle Type uint8_t (None, Relay, PWM, etc)
  • Spindle Max RPM float
  • Spindle RPM Min Float
  • Spindle PWM Off PWM uint16_t
  • Spindle PWM Min uint16_t
  • Spindle PWM Max Uint16_t
  • Spindle I/O output pin
  • Spindle I/O enable pin
  • Spindle I/O direction pin
  • Spindle TX pin
  • Spindle RX pin
  • Spindle TX Enable pin
  • Spindle modbus address uint8_t
  • Spindle Invert Output pin
  • Spindle Invert Enable pin
  • Spindle Invert Direction pin
  • Spindle Enable Piece wise Linear bool (what do we do about the pieces?)
• Custom Stuff
• Kinematics etc. (might not be practical, because custom compile likely required)

Grbl is already using [ESPxxx] for ESP3D and some other things. It might make sense to use the existing parsing for the '[' and use a...

[$SPINDLE_MAX_RPM]200.5

...format. It it could context switch on the '$'

• uint8_t (also used for bool)
• uint16_t
• uint32_t
• int32_t (signed are rare, so all will use this)
• float
• string
• Add more only as needed (maybe byte array)

The command line format "$key=value" is very familiar in many contexts - Classic GRBL (e.g. $100=80), PHP, JavaScript, TinyG - and is easy to read and type. The downside is that GRBL's existing parser for $ lines is an ad-hoc mess. However, detailed analysis of that existing parser shows that it's possible to make a simple systematic parser that handles the existing syntax while extending to named settings. Here's how:

The presence of a '$' as the first character of the line invokes the following procedure (described as pseudocode, but actual code is given later) that replaces system_execute_line():

• "Normalize" means to find the first unbroken sequence of non-white characters and convert them to uppercase.

Discard the leading '$' then search for a '=' in the line. If '=' is present

• Normalize the preceding characters as key. The following characters, with no whitespace modification are value
• Search for a match for key in the dollarSettings table (which maps strings to objects)
• If a match is found, call the object's set method with value as argument
• Otherwise, report an error

If '=' is absent:

• Normalized the remainder of the line as key
• Search for a match for key in the dollarCommands table (which maps strings to functions)
• If a match is found, call the function
• As an enhancement, if a Commands match is not found, look for a Settings match and display the value.
• Otherwise report an error

The object's set methods are responsible for conversion from the string form of value to internal data type, which may involve whitespace removal, validation, and any other ancillary error checking that may need to occur. The top level parser just splits the command line and dispatches based on the key. Common subroutines or inherited class methods can be used to handle conversion and validation situations that are common to multiple settings.

Similarly, commands that are only valid in specific states are responsible for checking that validity. It's not the parser's job.

Note that the value must not be so normalized like key, otherwise it will be impossible to store things like WiFi passphrases.

Here is some code (untested) to peruse:

#include <map>
#include <string>
#include <iterator>

// start points to a null-terminated string.
// Return the first substring that does not contain whitespace,
// converted to upper case.
char *normalize_key(uint8_t *start) {
  uint8_t c;

  // In the usual case, this loop will exit on the very first test,
  // because the first character is likely to be non-white.
  // Null ('\0') is not considered to be a space character.
  while (isspace(c = *start) && c != '\0') {
    ++start;
  }

  // start now points to either a printable character or end of string
  if (c == '\0') {
    return start;
  }

  // Having found the beginning of the printable string,
  // we now scan forward, converting lower to upper case,
  // until we find a space character.
  uint8_t end;
  for (end = start; (c = *end) != '\0' && !isspace(c); end++) {
    if (islower(c)) {
      *end = toupper(c);
    }
  }

  // end now points to either a whitespace character of end of string
  // In either case it is okay to place a null there
  *end = '\0';

  return start;
}

// This is for changing settings with $key=value .
// Lookup key in the dollarSettings map.  If found, execute
// the corresponding object's "set" method.  Otherwise fail.
setting_t do_setting(const uint8_t *key, uint8_t *value, uint8_t client) {

  std::string k = key;
  map<string, Setting_t>::iterator i = dollarSettings.find(k);
  if (i != dollarSetting.end()) {
    return i->second.set(value, client);
  }

  return (STATUS_INVALID_STATEMENT);
}

// This is for bare commands like "$RST" - no equals sign.
// Lookup key in the dollarCommands map.  If found, execute
// the corresponding command.
// As an enhancement to Classic GRBL, if the key is not found
// in the commands map, look it up in the dollarSettings map
// and display the current value.
setting_t do_command(const uint8_t *key, uint8_t client) {

  std::string k = key;
  map<string, Command_t>::iterator i = dollarCommands.find(k);
  if (i != dollarCommands.end()) {
    return i->second(client);
  }

  // Enhancement - not in Classic GRBL:
  // If it is not a command, look up the key
  // in the Settings map and display the value.
  std::string k = key;
  map<string, Setting_t>::iterator i = dollarSettings.find(k);
  if (i != dollarSetting.end()) {
    Setting_t s = i->second;
    display("$%s=%s\n", key, s.value_to_string(s.get()));
    return (STATUS_OK);
  }

  return (STATUS_INVALID_STATEMENT);
}

uint8_t system_execute_line(char* line, uint8_t client) {
  uint8_t *value = (uint8_t *)strchr(line, '=');

  if (value) {
    // Equals was found; replace it with null and skip it
    *value++ = '\0';
    do_setting(normalize_key(line), value, client);
  } else {
    // No equals, so it must be a command
    do_command(normalize_key(line), client);
  }
}

// The following table is used if the line is of the form "$key\n"
// The key value is matched against the string and the corresponding
// function is called with no arguments.
// If there is no key match an error is reported
typedef uint8_t (*Command_t)(void);
std::map<std::string, Command_t> dollarCommands = {
    { "$", report_normal_settings },
    { "+", report_extended_settings },
    { "G", report_gcode_modes },
    { "C", toggle_check_mode },
    { "X", disable_alarm_lock },
    { "#", report_ngc },
    { "H", home_all },
    { "HX", home_x },
    { "HY", home_y },
    { "HZ", home_z },
    { "HA", home_a },
    { "HB", home_b },
    { "HC", home_c },
    { "SLP", sleep },
    { "I", report_build_info },
    { "N", report_startup_lines },
};
// The following table if the line is of the form "$key=value".
// The object's set method is invoked with the value as argument.
// If no match is found in this table and the key is of the form N<number>,
// set_startup_line(number, value) is called; otherwise an error is reported

// Setting_t is a class

std::map<std::string, Setting_t> dollarSettings = {
    { "J", jog },
    { "I", build_info },
    { "RST", reset },
    { "0", steps_per_mm },
    { "1", max_rate },
    { "2", acceleration },
    { "3", max_travel },
    { "4", run_current },
    { "5", hold_current },
    { "6", microstepping },
    { "7", stallguard },
    { "10", status_mask },
    { "11", junction_deviation },
    { "12", arc_tolerance },
    { "13", report_inches },
    { "20", soft_limits },
    { "21", hard_limits },
    { "23", homing_dir },
    { "24", homing_feed },
    { "25", homing_seek },
    { "26", homing_debounce },
    { "27", homing_pulloff },
    { "30", spindle_max_speed },
    { "31", spindle_min_speed },
    { "32", laser_mode },
    { "100", x_steps },
    { "101", y_steps },
    { "102", z_steps },
    { "103", a_steps },
    { "104", b_steps },
    { "105", c_steps },
    { "110", x_max_rate },
    { "111", y_max_rate },
    { "112", z_max_rate },
    { "113", a_max_rate },
    { "114", b_max_rate },
    { "115", c_max_rate },
    { "120", x_acceleration },
    { "121", y_acceleration },
    { "122", z_acceleration },
    { "123", a_acceleration },
    { "124", b_acceleration },
    { "125", c_acceleration },
    { "130", x_max_travel },
    { "131", y_max_travel },
    { "132", z_max_travel },
    { "133", a_max_travel },
    { "134", b_max_travel },
    { "135", c_max_travel },
    { "N0", startup_line_0 },
    { "N1", startup_line_1 },
    { "STEPS_PER_MM", steps_per_mm },
    { "MAX_RATE", max_rate },
    { "ACCELERATION", acceleration },
    { "MAX_TRAVEL", max_travel },
    { "RUN_CURRENT", run_current },
    { "HOLD_CURRENT", hold_current },
    { "MICROSTEPPING", microstepping },
    { "STALLGUARD", stallguard },
    { "STATUS_MASK", status_mask },
    { "JUNCTION_DEVIATION", junction_deviation },
    { "ARC_TOLERANCE", arc_tolerance },
    { "REPORT_INCHES", report_inches },
    { "SOFT_LIMITS", soft_limits },
    { "HARD_LIMITS", hard_limits },
    { "HOMING_DIR", homing_dir },
    { "HOMING_FEED", homing_feed },
    { "HOMING_SEEK", homing_seek },
    { "HOMING_DEBOUNCE", homing_debounce },
    { "HOMING_PULLOFF", homing_pulloff },
    { "SPINDLE_MAX_SPEED", spindle_max_speed },
    { "SPINDLE_MIN_SPEED", spindle_min_speed },
    { "LASER_MODE", laser_mode },
    { "X_STEPS", x_steps },
    { "Y_STEPS", y_steps },
    { "Z_STEPS", z_steps },
    { "A_STEPS", a_steps },
    { "B_STEPS", b_steps },
    { "C_STEPS", c_steps },
    { "X_MAX_RATE", x_max_rate },
    { "Y_MAX_RATE", y_max_rate },
    { "Z_MAX_RATE", z_max_rate },
    { "A_MAX_RATE", a_max_rate },
    { "B_MAX_RATE", b_max_rate },
    { "C_MAX_RATE", c_max_rate },
    { "X_ACCELERATION", x_acceleration },
    { "Y_ACCELERATION", y_acceleration },
    { "Z_ACCELERATION", z_acceleration },
    { "A_ACCELERATION", a_acceleration },
    { "B_ACCELERATION", b_acceleration },
    { "C_ACCELERATION", c_acceleration },
    { "X_MAX_TRAVEL", x_max_travel },
    { "Y_MAX_TRAVEL", y_max_travel },
    { "Z_MAX_TRAVEL", z_max_travel },
    { "A_MAX_TRAVEL", a_max_travel },
    { "B_MAX_TRAVEL", b_max_travel },
    { "C_MAX_TRAVEL", c_max_travel },
    // Add additional named settings here
};