8. Logging data to the microSD card (Arduino) - ControlBits/EMIT GitHub Wiki

The EMIT development board, includes a microSD card socket to provide local data storage.

The microSD card can be used for many tasks such as storing temperature and humidity measurements in an environmental monitoring application (as covered in this guide), to buffer data while awaiting transmission via MQTT or to store hardware settings.

In this case, we're going to store our timestamp, temperature and humidity data to a .csv (comma seperated value) file called 'data.csv'.

8.1 Setting up the microSD card socket

The ESP32 Arduino SDK includes all of the basic SD card functional building blocks we need in the ‘SD’ and 'FS' for file system libraries. We just need to configure the GPIO pin for Chip select.

As the EMIT interface to the SD card is 'SPI', we will need to also include the SPI library along with 'SD' and 'FS'.

#include "FS.h"
#include "SD.h"
#include <SPI.h>

EMIT's microSD Card socket is connected to the ESP32's SPI port on ‘Slot 2’ (i.e. SCK = GPIO18, CS = GPIO5, MISO = GPIO19 & MOSI = GPIO23). There is also a hardware ‘Card Detect’ (CD) switch connected to GPIO4 pin.

Setting up the microSD card socket's Chip select pin is handled by the machine.SDCard() function:

// Initialize SD card
SD.begin(SD_CS);

Where SD_CS is defined for readability to GPIO4.

// Define CS pin for the SD card module
#define SD_CS 4

Next, we need to define the name of the file we wish to write to on the microSD card:

// name of file (data.csv) to save data to on SD Card
const char* fileName = "/data.csv";

We also need to define a variable 'sd_mounted' to hold the status of the microSD card mounting process:

bool sd_mounted = false;

Finally, we need to create a couple of functions ... the first will to attempt to mount the microSD card to the ESP32's operating system. With some error handling, our 'mount_sd()' function is:

// define function to mount SD card
void mount_sd(void){
 // initialize sd_mounted flag to false.
 sd_mounted=false;
 Serial.println("Initializing SD card...");
 if (!SD.begin(SD_CS)) {
   // print error message to Serial Monitor.
   Serial.println("ERROR - SD card initialization failed!");
   return;    // init failed
 }
 // print success message to Serial Monitor.
 Serial.println("SD Card mounted!");
 // set sd_mounted to true
 sd_mounted=true;
}

We also need a function, 'write_to_sd(data)', to write data to the microSD card and append_to_sd to append new data to the file without overwriting old data.

// define function for writing data to SD Card
void write_to_sd(const char * data) {
 Serial.println("Writing to file");
 // open file 'fileName'  for writing
 File file = SD.open(fileName, FILE_WRITE);
 if(!file) {
   Serial.println("Failed to open file for writing");
   return;
 }
 // write data to file
 if(file.print(data)) {
   Serial.println("File written");
 } else {
   Serial.println("Write failed");
 }
 // close opened file (fileName)
 file.close();
}

8.2 Using the microSD card

We're now ready to use our new microSD card functions. First we need to mount the microSD card using:

// mount the microSD card
mount_sd();

Once we have checked that the microSD card is successfully mounted (i.e. sd_mounted == true), all we need to do is pass the data we wish to write to the microSD card as a string to our append_to_sd() function.

But first we will check if the 'data.csv' already exists and if not we create it and write the csv header to it timestamp,temperature(°C),humidity(%)

 // If the data.txt file doesn't exist
 // Create a file on the SD card and write the data labels
 File file = SD.open(fileName);
 if(!file) {
   Serial.println("File doesn't exist");
   Serial.println("Creating file...");
   write_to_sd("timestamp,temperature(°C),humidity(%)\r\n");
 }
 else {
   Serial.println("File already exists"); 
 }
 file.close();

Then we're going to concatenating a comma separated string containing our timestamp (get_time_str()), temperature (tempC) and humidity (humidity) data as follows:

First we define a variable holding the csv string.

char csvString[60];

Then we will concatenate the string from the timestamp, temperature (tempC) and humidity.

snprintf(csvString,60,"%s,%f,%f\r\n",get_time_str(),tempC,humidity);

Finally we append the string to the end of our .csv file:

 // if SD card is mounted, write timestamp, temperature and humidity data to it.
 if (sd_mounted == true)
   // open the file for appending.
   // record new data to the SD card file.
   append_to_sd(csvString);

And that's all there is to it ... now EMIT can record it's measurements to it's microSD card!

For reference, the completed Sketch is:

// File system library
// for working with file on file system
#include "FS.h"
// SD card library
#include "SD.h"
// SPI library for interfacing the SD card.
#include <SPI.h>
// WiFi library is essential for connecting to WIFI
#include <WiFi.h>
// contains the TCP client object needed to establish a MQTT connection with the MQTT broker.
#include <WiFiClient.h>
// MQTT client C++ implementation library
#include <PubSubClient.h>
// time library sync EMIT ESP32 clock with a NTP server
#include "time.h"
// include the Adafruit DHT libraries to use its functions reading the temperature and humidity from the DHT22 sensor.
#include "DHT.h"
 
 
// GPIO Definitions
#define EMIT_RED_LED 16
#define EMIT_GREEN_LED 17
#define EMIT_RELAY    26
 
#define EMIT_DHT_PIN 14     // Digital pin connected to the DHT sensor
 
#define DHTTYPE DHT22   // DHT 22  (AM2302), AM2321
 
// Define CS pin for the SD card module
#define SD_CS 4
 
// Initialize DHT sensor.
DHT dht(EMIT_DHT_PIN, DHTTYPE);
 
// define Wi-Fi settings
#define WIFISSID "your-WiFi-SSID-goes-here"
#define WIFIPASSWORD "your-WiFi-password-goes-here"
 
// we will request the time from pool.ntp.org
const char* ntpServer = "pool.ntp.org";
// The gmtOffset_sec variable defines the offset in seconds between your time zone and GMT
const long  gmtOffset_sec = 0;
// defines the offset in seconds for daylight saving time. It is generally one hour, that corresponds to 3600 seconds
const int   daylightOffset_sec = 3600;
 
// array holding the json string holding the timestamped readings
char JSONstring[150];
 
 
// MQTT
const char* deviceUUID = "f1f2c71d-3164-4edc-a0a4-75aa172912d0";
const char* mqtt_broker = "public.mqtthq.com";
int mqtt_port = 1883;
 
WiFiClient emitClient;
PubSubClient mqttclient(emitClient);
 
long lastReconnectAttempt = 0;
 
// define relayState string and set to "OFF"
String relayState = "OFF";
// temperature set point for relay (Celsius)
float tempSet = 21.0;
// relay control loop hysteresis (Celcius)
float hysteresis = 0.5;
 
// name of file (data.csv) to save data to on SD Card
const char* fileName = "/data.csv";
 
// flag to report if the filesystem is mounted successfully.
bool sd_mounted = false;
 
 
// variable holding the SD card record csv formatted string.
char csvString[60];
 
 
void mqttclient_callback(char* topic, byte* payload, unsigned int length) {
 // handle message arrived
}
 
// * handles the mqtt client connection.
// * Attempt to connect to the mqtt broker and if it fails.
// * or if it's not connected or failed to connect it initiates a timer to reconnect in 5 seconds.
// * call mqttclient.loop which needed to be called continuously in loop for the mqtt client to
// function probably as it handles the keep alive packets to the broker and checks for a received message, etc..
void mqttclient_connection(void) {
 // check if the MQTT client is connected or not
 if (!mqttclient.connected()) {
   // not connected then start a timer to try to reconnect after 5 seconds
   long now = millis(); // now holds the current time.
   // check if 5 seconds is elapsed.
   if (now - lastReconnectAttempt > 5000) {
     lastReconnectAttempt = now; // update the last connection attempt time with the current time to try again after 5 seconds if failed
     // Attempt to reconnect
     // try to connect to the broker..
     mqttclient.connect(deviceUUID);
     if (mqttclient.connected()) {
       // only if connected successfully stop the timer
       lastReconnectAttempt = 0;
     }
   }
 } else {
   // Client connected then calls the mqttclient.loop that maintains the keep alive messages with the broker, checks for a received message etc..
   mqttclient.loop();
 }
}
 
void wifi_connect(void){
 // change the wifi interface mode to station
 WiFi.mode(WIFI_STA);
 
 // attempt a WiFi connection to the router with credentials of WIFISSID and WIFIPASSWORD
 WiFi.begin(WIFISSID, WIFIPASSWORD);
 
 // wait for the connection to be established.
 while (WiFi.status() != WL_CONNECTED){
     // flash the Green LED indication a wifi connection attempt.
     // turn on the Green LED
     digitalWrite(EMIT_GREEN_LED , HIGH);
     // print 'trying..' message to Serial Monitor
     Serial.print("trying WiFi connection ");
     Serial.println(WIFISSID); 
 
     // wait 1 second (1000 milliseconds)
     delay(1000);
 
     // turn off the Green LED
     digitalWrite(EMIT_GREEN_LED , LOW);
 
     // wait 2 second (2000 milliseconds)
     delay(2000);     
 }
 // if connected ...
 // turn on the Green LED
 digitalWrite(EMIT_GREEN_LED , HIGH);
 
 // print 'WiFi connection successful' message to Serial Monitor
 Serial.println("WiFi connection successful") ;
 
 // print WiFi network settings (inc. IP Address) to Serial Monitor.
 Serial.println("IP address: ");
 Serial.println(WiFi.localIP());
}
// define function for getting local time as string
// returns a pointer to a string containing the formatted time.
char* get_time_str(void){
 // string holding well formatted time.
 static char my_time_string[60];
 // structure holding the time
 // definition is as follow
 //     struct tm
 //    {
 //      int tm_sec;  // seconds from 0 to 59
 //      int tm_min;  // minutes from 0 to 59
 //      int tm_hour; // hours from 0 to 23
 //      int tm_mday; // day of the month from 1 to 30
 //      int tm_mon;  // month of the year from 0 to 11
 //      int tm_year; // number of years from 1900
 //      ... other variables not in the scope of the tutorial..
 //    };
 struct tm my_time;
 // read the time into the timeinfo structure.
 if(!getLocalTime(&my_time)){
   Serial.println("Failed to obtain time");
   return NULL;
 }
 snprintf(my_time_string, // string to add the formatted time to.
          60, // string length to prevent overflow
          "%04d-%02d-%02d %02d:%02d:%02d", // time string format
          (my_time.tm_year + 1900),
          (my_time.tm_mon  + 1),
           my_time.tm_mday,
           my_time.tm_hour,my_time.tm_min,my_time.tm_sec);
 
 // return the formatted time string
 return my_time_string;    
}
 
// define function to mount SD card
void mount_sd(void){
 // initialize sd_mounted flag to false.
 sd_mounted=false;
 Serial.println("Initializing SD card...");
 if (!SD.begin(SD_CS)) {
   // print error message to Serial Monitor.
   Serial.println("ERROR - SD card initialization failed!");
   return;    // init failed
 }
 // print success message to Serial Monitor.
 Serial.println("SD Card mounted!");
 // set sd_mounted to true
 sd_mounted=true;
}
// define function for writing data to SD Card
void write_to_sd(const char * data) {
 Serial.println("Writing to file");
 // open file 'fileName'  for writing
 File file = SD.open(fileName, FILE_WRITE);
 if(!file) {
   Serial.println("Failed to open file for writing");
   return;
 }
 // write data to file
 if(file.print(data)) {
   Serial.println("File written");
 } else {
   Serial.println("Write failed");
 }
 // close opened file (fileName)
 file.close();
}
 
// define function for append data to SD Card
void append_to_sd(const char * data) {
 Serial.println("Appending to file ");
 // open file 'fileName'  for appending
 File file = SD.open(fileName, FILE_APPEND);
 if(!file) {
   Serial.println("Failed to open file for appending");
   return;
 }
 // write data to file
 if(file.print(data)) {
   Serial.println("Data appended");
 } else {
   Serial.println("Append failed");
 }
 // close opened file (fileName)
 file.close();
}
void setup() {
 // Enable the serial terminal
 // Useful for debugging messages
 Serial.begin(115200); 
 // configure RED LED GPIO to be OUTPUT
 // so that we would control the LED (ON , OFF)
 pinMode(EMIT_RED_LED , OUTPUT);
  // on the beginning we will turn OFF the RED LED
 digitalWrite(EMIT_RED_LED , LOW);
 
 // configure Relay GPIO to be OUTPUT
 // so that we would control the Relay (ON , OFF)
 pinMode(EMIT_RELAY , OUTPUT);
  // on the beginning we will turn OFF the Relay
 digitalWrite(EMIT_RELAY , LOW);
  // configure Green LED GPIO to be OUTPUT
 // so that we would control the LED (ON , OFF)
 pinMode(EMIT_GREEN_LED , OUTPUT);
  // on the beginning we will turn OFF the GREEN LED
 digitalWrite(EMIT_GREEN_LED , LOW);
  // initialize the DHT communication
 dht.begin();
 
 // attempt a wifi connection
 wifi_connect();
 
 // wait 5 seconds ( 5000 milliseconds ) for network to settle
 delay(5000);
  // synchronise time with the NTP server.
 configTime(gmtOffset_sec, daylightOffset_sec, ntpServer);
 
 // initialize the MQTT client with the broker name and port number
 mqttclient.setServer(mqtt_broker, mqtt_port);
 // set the reception callback of the MQTT client to be called by the client stack on reception of a new message.
 mqttclient.setCallback(mqttclient_callback);
 
 // mount SD card filesystem..
 mount_sd();
 
 // If the data.txt file doesn't exist
 // Create a file on the SD card and write the data labels
 File file = SD.open(fileName);
 if(!file) {
   Serial.println("File doesn't exist");
   Serial.println("Creating file...");
   write_to_sd("timestamp,temperature(°C),humidity(%)\r\n");
 }
 else {
   Serial.println("File already exists"); 
 }
 file.close();
}
 
void loop() {
 // handle the whole MQTT connection
 mqttclient_connection();
 //Wait for 5 seconds ( 5000 milliseconds )
 delay(5000);
 
 // turn the red LED onma
 digitalWrite(EMIT_RED_LED , HIGH);
 
 // Print a debugging message on the Serial Terminal indicating the start of the read attempt.
 Serial.println("Reading AM2302 ...");
 // Reading temperature or humidity takes about 250 milliseconds!
 // Sensor readings may also be up to 2 seconds
 float humidity = dht.readHumidity();
 // Read temperature as Celsius (the default)
 float tempC = dht.readTemperature();
 // Read temperature as Fahrenheit (isFahrenheit = true)
 float tempF = dht.readTemperature(true);
  // turn the red LED off indicating the end of the read cycle
 digitalWrite(EMIT_RED_LED , LOW);
 
 if(tempC >= tempSet){
   // turn relay OFF
   digitalWrite(EMIT_RELAY , LOW);
   // set relayState string to "OFF"
   relayState = "OFF";   
    
 }
  if ((tempC > (tempSet - hysteresis)) && relayState == "OFF"){
   // turn relay OFF
   digitalWrite(EMIT_RELAY , LOW);
 }
 else{
   // turn relay ON
   digitalWrite(EMIT_RELAY , HIGH);
   // set relayState string to "ON"
   relayState = "ON";
 }
  snprintf(JSONstring,150,
         "{\"TimeUTC\":\"%s\",\"TemperatureC\":\"%f\",\"Humidity\":\"%f\",\"TempSetPointC\":\"%f\",\"Relay\":\"%s\"}",
         get_time_str(),tempC,humidity,tempSet,relayState);
 
       
 // Print the read values on the Serial Monitor.
 Serial.println(JSONstring);
 
 // only call the publish function if the client is really connected to the broker
 if(mqttclient.connected())
   mqttclient.publish(deviceUUID, JSONstring);
 else
   // if not connected the mqtt_connection function will try to recover shortly on the next loop
   Serial.println("failed to publish, no connection...");
 
 snprintf(csvString,60,"%s,%f,%f\r\n",get_time_str(),tempC,humidity);
 
 // if SD card is mounted, write timestamp, temperature and humidity data to it.
 if (sd_mounted == true)
   // open the file for appending.
   // record new data to the SD card file.
   append_to_sd(csvString);
 
}

Now data is being logged to the SD card on CSV format like below.

data.csv

8. Logging data to the microSD card (Arduino) - ControlBits/EMIT GitHub Wiki

8.1 Setting up the microSD card socket

8.2 Using the microSD card

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️