How to get CAISO Day Ahead Electricity Market Price Data - SCU-Smart-Grid-CPS/smart-grid-energy-simulation-research GitHub Wiki

Version 1.0 - Stable, Full Functionality
Brian Woo-Shem
Last updated: 2021-07-28

Automatically get any number of days CAISO wholesale price data in Python. Simply run this program with type of data and what dates as shown below, and it will generate a .csv with wholesale price data in a single column at 5 minute timesteps that can be plugged directly into UCEF and the optimizer script.

Outputs csv with file name in format: WholesaleRealTime_2019-12-21_2020-3-30.csv

New script getWholesaleCAISO.py

SYNTAX:

getWholesaleCAISO.py <data_type> <year> <month> <day> <number_of_days> <optional_options>

<data_type>  d for day ahead, r for real time
<year>       four digit year of first day to get (eg. 2020)
<month>      month of the first day to get (eg. January, Jan, or 1)
<day>        day of month for the first day (eg. 21)
<number_of_days>    how many days to get data for. 1 = only get the day specified
<optional_options>  Additional parameters, optional
   -v      verbose: provide extra outputs for user readability
   -d      debug: provide detailed outputs for debugging
   -leap   include leap years in date calculation
   -s      Server Safe mode; adds delay to avoid crashing server but takes longer.
   -fast   fast unsafe: removes all time delays for fastest data acquisition. WARNING: you are responsible if this mode breaks something or violates the CAISO Terms of Use.

For licensing info, run getWholesaleCAISO.py -l

Required Modules: beautifulsoup4, urllib3, numpy, os_sys, lxml, requests

Example Input: Returns week starting Aug 3, 2020 with day-ahead values

getWholesaleCAISO.py d 2020 8 3 7

Version 0.9 - Stable, limited functionality, mostly manual operation
Brian Woo-Shem
Last updated: 2021-07-21

CAISO Day Ahead Market rates determine pricing in the optimization simulation. Simulation program requires data for every 5 minute timestep for the entire simulation in one long column, but the main data source only gives us a graph one day at a time. This process can quickly obtain and convert the data format.

Use this link format, but change the month, day, and year as indicated. For single digit days, use 01, 02,...

www.caiso.com/Documents/Day-AheadDailyMarketWatchMMMDD-YYYY.html

On the first graph, hover over so the PGAE line is selected. Notice that it provides the price by hour in the pop up box.

Right click while still selecting this line, and click Inspect

In the Inspector Console window, go to the Sources tab and click Day-AheadDailyMarketWatch… on the left panel.

Scroll to line 447, it should say var lmp_pgae_ifm and then an array

Copy this line into a text file. Double check that the numbers match the PGAE line on the first graph.

Repeat for as many days as needed. Once complete, the text file will look like this:

Remove all of the var lmp_pgae_ifm = [ and ], text. Get all the numbers separated by commas only. Use find and replace for more than a few days.

Should now look like this:

Save the file as a .csv

Use the hourlyTo5min.py code to convert it to 5 minute data in columns needed by UCEF. Result looks like this:

⚠️ **GitHub.com Fallback** ⚠️