Continued debugging the issues with the example code on m2 mac. Got as far as replacing the function isfinite() with finite(). Source code was updated through a PR made during general VIP meeting. Still have segfault issue.

Began researching powerflow equations. Basically kirchoff analogue for AC circuits. Instead of being ODEs in a single variable system, they are PDEs in a multivariate system. Because of this, to solve powerflow equations, estimation methods are employed, with the most common being Range-Kutta. Gridlabd uses something similar to simulate powerflow. Using purely estimation without taking into account other factors can be flawed, due to changes in the system. I did some research on alternatives to traditional methods, and found that Graphical Neural Networks (GNNs) can be used to approximate PDEs while maintaining the dynamics of the system. GNNs are also able to take into account actual data, which would account for failures in the grid, etc. More information can be found here and here.

Also discovered the mac m2 issue is on HELICS/gridlabd's end (probably), and to fix it would mean going through the source code. For now, will continue researching without being able to run simulations.

Began work on the midterm presentation, including compiling research, etc. Will compile the research I did last week into a slide-based format.

Also decided on a common structure for the device simulations. Will probably work out the exact ADT next week through draw.io in a subteam meeting, and begin the implementation after deciding exactly what interfaces/abstract classes will be needed.

We worked on the midterm presentation, and split up what we needed to do according to what people worked on. I started looking in to how I could implement some of my research (ie pytorch libraries, etc), and looked at documentation on that

• We started working on the midterm paper, because it was past finals week
• I worked on the powerflow section
• I continued doing my own research on how I could utilize Graph Convolution Networks to simulate the voltage differences
• I needed to hold off on this research for a little bit because of more pressing matters on the team end Current GNN code

import torch
import torch.nn.functional as F
from torch_geometric.nn import GCNConv

# Define a Graph Convolutional Network layer
class PowerFlowGCN(torch.nn.Module):
    def __init__(self):
        super(PowerFlowGCN, self).__init__()
        self.conv1 = GCNConv(1, 16)
        self.conv2 = GCNConv(16, 1)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = self.conv2(x, edge_index)
        return x

if (__name__ == "__main__"):
    # Define a simple electrical grid as a graph
    # In this example, let's consider a small grid with 4 nodes (buses) and 4 edges (power lines)
    edge_index = torch.tensor([[0, 1, 1, 2, 2, 3, 3, 0], [1, 0, 2, 1, 3, 2, 0, 3]], dtype=torch.long)
    num_nodes = 4

    # Define initial node features (voltage magnitudes)
    initial_voltage = torch.tensor([1.0, 1.0, 1.0, 1.0], dtype=torch.float).view(-1, 1)
    x = initial_voltage.clone()

    # Instantiate the GCN model
    gcn_model = PowerFlowGCN()

    # Set the number of iterations for solving power flow equations
    num_iterations = 10

    # Simulate power flow iterations
    for _ in range(num_iterations):
        # Forward pass through the GCN model
        x = gcn_model(x, edge_index)
        
        # Clamp voltage magnitudes to ensure they remain positive
        x = F.relu(x)

    # Display the final voltage magnitudes
    print("Final Voltage Magnitudes:")
    print(x)

• Xin spit us up into subteams in the group, which meant I would focus more on the entire group's goals as opposed to my separate part.
• I got assigned to Co-simulation and voltage regulation. Because the alternative methods for powerflow was going to slowly, I decided to look further into gridlab-d with the rest of my team

• Our group started planning out what we needed to do for the simulation
• Agreed to meet over thanksgiving break to work on our tasks
• This was also a big exam week, so I didn't do much besides that

• Amani and I met over Thanksgiving break. We took a look over the gridlab-d files, and worked on writing config files for the devices.
• We also went over what python files we needed to edit (the integrations and stuff)
• I also started working on a frontend for the powerflow simulation stuff I had coded up Frontend Element For storing volatges

import { useEffect, useState } from "react"

export function VoltageLabel({initVoltage, voltageMap, setVoltageMap, id, nodes, setNodes}) {
    const [voltage, setVoltage] = useState(initVoltage);
    const [btnState, setBtnState] = useState(true);
    useEffect(() => {
            voltageMap[id] = voltage;
            setVoltageMap(voltageMap);
            console.log(voltageMap);
    }, [voltage])

    const handleBtnClick = () => {
        if (btnState) {
            const val = document.getElementById(`inpt${id}`).value;
            setVoltage(val)
        }
        setBtnState(!btnState)
    }

    const handleDelete = () => {
        const newNodes = nodes.filter((node) => node.id !== id);
        console.log(nodes)
        delete voltageMap[id];
        setVoltageMap(voltageMap);
        setNodes(newNodes)
    }

    return (
        <div style={{fontWeight: "550"}}>
            <span style={{marginRight: "-20%"}}>Bus Node</span>
            <button style={{backgroundColor: "#e63946", float: "right"}} onClick={handleDelete}>-</button>
            <p></p>
            {btnState ? <input type="text" id={`inpt${id}`} style={{width: "10%", height: "5%", textAlign: "center"}} defaultValue={voltage}></input> : <span>{voltage} </span>}V
            <p></p>
            <button onClick={handleBtnClick}>{btnState ? <div>Done</div> : <div>Edit</div>}</button>
        </div>
    )
}

• I uploaded everything I needed to for git (the semester's work)
• Throughout the course of this VIP, I got some exposure to pytorch for the ML side of this project (I had previously only used tensorflow)
• This was because pytorch has built-in support for GNNs
• The code took me a little longer to make than usual because I needed to learn how to use the libraries
• I already had some experience in React, so it was pretty easy to make the frontend

We had our first non-introductory meeting of this semester. Because there were an influx of new students to the VIP, some of the other roll over VIP students and I went over what we did last semester in the device modeling subteam. We also decided on a meeting time for outside of VIP subteam meetings (Sunday @ 1PM) and started our initial presentation.

We split into subteams, and then started planning out the material for our initial presentation. This involved task division within our subteam (deciding who was doing cosimulation v networking), goal-settting for each task for the semester, and a timeline for each. Because about 50% of our subteam is new, we allocated 20 days of training time for them.

Most of the tasks this semester are refining what we learned last semester by brining them into the actual context of the simulation. A lot of last semester was spent wrangling with HELICS not working on various platforms, but with this ironed out, we should be able to hit the ground running this winter.

This week was more of set up of environments. Some of the newer members of the VIP were having some trouble with nix installation. There was also a bit more discussion on subteam selection. We should aim to hit the ground running next meeting.

I wasn't able to make much progress as the first wave of midterms came in. I hope to be able to make up this deficit in the coming weeks

We worked on the midterm paper. It was interesting being a leader this time around with the development of the paper being a second semester student, giving advice to people asking for it and helping to organize meetings. It is a little bit more work without Xin organizing everything.

Not much in terms of technical work got done this week. It was mainly presentation preperation. In addition to completing my slides I aided some of the newwer members, like Dhruv, in preparation for the actual presentation (what to expect in terms of questions/flow of the project, etc. It is really satisfying to see this coming together like last semester.

After this first stage of the VIP, where we trained the new members, I am looking forward to the boost of progress that this growth will provide our subteam and the project as a whole.

After midterm exam season, I continued working on the powerflow-ml code. There was some issues with the gnn library looking at the metadata, so I will aim to look at that over the next few weeks. I also started looking into alternative models to GCN such as GAT (graph attention networks) and GIN (Graph Isomorphism Network).

import torch
import torch.nn.functional as F
from torch_geometric.nn import GATConv

class PowerFlowGAT(torch.nn.Module):
    def __init__(self):
        super(PowerFlowGAT, self).__init__()
        self.conv1 = GATConv(1, 16, heads=4, dropout=0.5)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index)
        x = F.elu(x)
        return x


if (__name__ == "__main__"):
    # Define a simple electrical grid as a graph
    edge_index = torch.tensor([[0, 1, 1, 2, 2, 3, 3, 0], [1, 0, 2, 1, 3, 2, 0, 3]], dtype=torch.long)
    num_nodes = 4

    # Define initial node features (voltage magnitudes)
    initial_voltage = torch.tensor([1.0, 1.0, 1.0, 1.0], dtype=torch.float).view(-1, 1)
    x = initial_voltage.clone()

    # Instantiate the GAT model
    gat_model = PowerFlowGAT()

    # Set the number of iterations for solving power flow equations
    num_iterations = 10

    # Simulate power flow iterations
    for _ in range(num_iterations):
        # Forward pass through the GAT model
        x = gat_model(x, edge_index)
        
        # Clamp voltage magnitudes to ensure they remain positive
        x = torch.relu(x)

    # Display the final voltage magnitudes
    print("Final Voltage Magnitudes (GAT):")
    print(x)

import torch
import torch.nn.functional as F
from torch_geometric.nn import GINConv

class PowerFlowGIN(torch.nn.Module):
    def __init__(self):
        super(PowerFlowGIN, self).__init__()
        self.conv1 = GINConv(torch.nn.Sequential(torch.nn.Linear(1, 16), torch.nn.ReLU(), torch.nn.Linear(16, 16)))
        self.conv2 = GINConv(torch.nn.Sequential(torch.nn.Linear(16, 16), torch.nn.ReLU(), torch.nn.Linear(16, 1)))

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = self.conv2(x, edge_index)
        return x


if (__name__ == "__main__"):

    # Define a simple electrical grid as a graph
    edge_index = torch.tensor([[0, 1, 1, 2, 2, 3, 3, 0], [1, 0, 2, 1, 3, 2, 0, 3]], dtype=torch.long)
    num_nodes = 4

    # Define initial node features (voltage magnitudes)
    initial_voltage = torch.tensor([1.0, 1.0, 1.0, 1.0], dtype=torch.float).view(-1, 1)
    x = initial_voltage.clone()

    # Instantiate the GIN model
    gin_model = PowerFlowGIN()

    # Set the number of iterations for solving power flow equations
    num_iterations = 10

    # Simulate power flow iterations
    for _ in range(num_iterations):
        # Forward pass through the GIN model
        x = gin_model(x, edge_index)
        
        # Clamp voltage magnitudes to ensure they remain positive
        x = torch.relu(x)

    # Display the final voltage magnitudes
    print("Final Voltage Magnitudes (GIN):")
    print(x)

I found that the issue was the way I was passing in tensors into tensorflow, so I changed over the way the tensors are shaped. The only problem is finding more data. While training on the existing gridlab-d simulation may be a good start, it would never surpass the simulations.

from flask import Flask, request, jsonify
import torch
import torch.nn.functional as F
from gnn import PowerFlowGCN
from gat import PowerFlowGAT
from gin import PowerFlowGIN
from json import loads
from flask_cors import CORS, cross_origin
from enum import Enum

app = Flask(__name__)
cors = CORS(app)
app.config['CORS_HEADERS'] = 'Content-Type'
gcn_model = PowerFlowGCN()
gat_model = PowerFlowGAT()
gin_model = PowerFlowGIN()

class ModelMode(Enum):
    GCN = 1
    GAT = 2
    GIN = 3


@app.route("/", methods=["POST", "GET"])
@cross_origin()
def callModel():
    if (request.method == "GET"):
        return dumps({
            "hello": "world"
        })
    json = loads(request.data.decode("utf-8"))
    print(json)
    edge_index = torch.tensor([json["src"],json["dest"]], dtype=torch.long)
    initialVoltage = torch.tensor(json["voltages"], dtype=torch.float).view(-1, 1)
    x = initialVoltage.clone()

    mode = json.get("mode", ModelMode.GCN)

    for i in range(json["steps"]):
        # Forward pass through the GCN model
        x = None
        
        if (mode == ModelMode.GAT):
            x = gat_model(x, edge_index)
        elif (mode == ModelMode.GIN):
            x = gin_model(x, edge_index)
        else:
            x = gcn_model(x, edge_index)
        
        # Clamp voltage magnitudes to ensure they remain positive
        x = F.relu(x)
    print(x.tolist())
    return {
        "voltages": x.tolist()
    }

if (__name__ == "__main__"):
    app.run(port=8000)

With the new year starting, we split into groups again with simulation and ROS, and got an influx of new members. Since this was our largest VIP subgroup yet, we had some trouble scheduling an outside meeting time. We also divided up work for the kickoff slides, but I was not able to attend the meeting.

After breaking up into subteams, we got to work on the kickoff presentation. I was unable to attend the first subteam meeting because my family had come over. I got my slides done and presented the task and goals slides for the device modeling subteam as well as the monitoring goal slide. I presented the goal slide because Dhruv had a conflict with the meeting time.

I started working on powerflow-ml for this semester, and started looking at different potential sources for data that I could train/test the current models I have on. I have not found a suitable dataset yet, but I will continue looking.

We started working on our midterm papers with the rest of our subteam. Due to the hurricane, our meeting was online. During the meeting, some concerns about attendance of the Sunday meetings was raised. I have missed one without notice this semester, so I will try to do my best with attending them for the rest of the semester. During this subteam meeting less than half of the VIP attended, so we were unable to split up the paper sections effectively. Also, there are unknowns like who has dropped the VIP etc. We have to follow up with Sam to figure out how to divide the labor further.

During our Wednesday meeting for the paper, a lot of members were absent, but we were able to get a good bit of work on the paper done. Since the deadline got moved to Friday, I was able to add more of what I wanted to the paper, but there was some confusion on what should go in which section.

Most of this week was spent working on the presentation. I didn't make much progress on the technical side of things because I had a couple of interviews and tests this week.

This week, I focused on searching for datasets that could enhance my models by introducing new testing conditions. In my search, I came across GridSTAGE, which looked promising due to its relevance to grid-based modeling. I spent some time examining the dataset’s structure and content to see how well it aligns with my models' requirements. However, initial analysis showed that GridSTAGE's format differs considerably from the inputs my models currently accept. For now, I’m cataloging the data’s key attributes and assessing its potential, while keeping an eye out for other datasets that might be easier to integrate.

After a closer look at the GridSTAGE dataset, I began a deeper assessment to see if adapting my models to accept this data would be feasible. I documented the primary challenges, particularly the need for substantial transformations in the data format. Given these challenges, I’ve started exploring alternative datasets that may have structures more compatible with my models, aiming to avoid extensive reworking of existing code. This week’s focus remains on identifying a data source that could align more directly with my models' current configurations.

I also looked at the other models that I had researched earlier in the semester in the hopes that they may be able to work better. I think by combing through the cited works section I should be able to find some datasets to test on.

This week, I set up a preliminary data preparation pipeline that could handle a variety of datasets without significant changes. I’m creating a flexible framework that will allow me to process incoming data with minimal adjustments, which will be useful in the future if I decide to incorporate GridSTAGE or other, similarly structured datasets. GridSTAGE presents a little bit of a challenge as its a simulation tool. Hopefully I can find more simplified datasets out there or investigate various cleaning techniques.

My best bet for finding a dataset is through the HELF paper I looked at earlier. I'll take more time when looking at the datasets.

I was unable to do too much work this week as I had to catch up on work from other classes, hopefully I can get back on track.

This week, I focused on working with the Holomorphic Embedding Load Flow (HELF) method for simulating power flow. I implemented some initial steps for the holomorphic voltage expansion by iterating over the holomorphic coefficients of the voltage. During this process, I encountered some NaN values in the computation, which seemed to stem from improper initialization of coefficients. After adjusting the initialization and adding some checks for zero-division errors, I was able to proceed with refining the computation steps. I also explored implementing the holomorphic embedding technique in PyTorch, using complex-valued tensors for voltage and power values. The goal is to use this method to simulate and analyze power flow more accurately.

In continuing my work with HELF, I refined the computation steps to improve the stability of the power flow calculations. The NaN issue I encountered earlier was linked to improper initialization and zero-division errors in the voltage coefficients. After refining the initialization process and adding checks to handle edge cases, the computations became more stable. I also focused on enhancing the accuracy of the voltage calculations across different grid configurations.

I have a implemented version in the wiki:

import torch
import torch.nn as nn

class HELFNet(nn.Module):
    def __init__(self, n_buses):
        """
        Initialize the HELF network.
        
        Args:
            n_buses: Number of buses in the power grid.
        """
        super(HELFNet, self).__init__()
        self.n_buses = n_buses

    def forward(self, y_bus, s, v_slack):
        """
        Perform the forward propagation using the HELF method.
        
        Args:
            y_bus: Complex-valued admittance matrix (Y-bus).
            s: Complex power injections (P - jQ).
            v_slack: Slack bus voltage (complex scalar).

        Returns:
            Complex-valued voltage solutions for all buses.
        """
        # Step 1: Initialize
        n = self.n_buses
        v = torch.zeros((n,), dtype=torch.cfloat)  # Initialize complex voltages
        v[0] = v_slack  # Set the slack bus voltage

        # Initialize holomorphic coefficients
        v_h = torch.zeros((n, n), dtype=torch.cfloat)  # Holomorphic voltage expansion
        v_h[:, 0] = v  # Base case: voltage coefficients

        # Step 2: Iteratively compute holomorphic coefficients
        for k in range(1, n):
            for i in range(1, n):  # Skip slack bus
                sum_term = torch.sum(y_bus[i, :] * v_h[:, k - 1])
                if torch.abs(v_h[i, k - 1]) < 1e-8:
                    v_h[i, k - 1] = 1e-8  # Small epsilon to avoid NaN
                v_h[i, k] = -s[i] / v_h[i, k - 1] - sum_term

        # Step 3: Analytic continuation to approximate voltages
        indices = torch.arange(1, n + 1, dtype=torch.float)  # Use real-valued arange
        indices = indices.to(torch.cfloat)  # Cast to complex
        for i in range(1, n):
            v[i] = torch.sum(v_h[i, :] / indices)

        return v


# Example Usage
if __name__ == "__main__":
    # Example Y-bus matrix (Admittance matrix)
    y_bus = torch.tensor(
        [[10+5j, -5j, 0, 0, 0],
         [-5j, 10+5j, -5j, 0, 0],
         [0, -5j, 10+5j, -5j, 0],
         [0, 0, -5j, 10+5j, -5j],
         [0, 0, 0, -5j, 10+5j]], 
        dtype=torch.cfloat
    )

    # Example power injections (complex P - jQ)
    s = torch.tensor([0+0j, -1-0.5j, -2-1j, -1.5-0.8j, -1-0.4j], dtype=torch.cfloat)

    # Slack bus voltage
    v_slack = torch.tensor(1.0 + 0j, dtype=torch.cfloat)

    # Initialize and run HELFNet
    helf_net = HELFNet(n_buses=5)
    voltages = helf_net(y_bus, s, v_slack)

    print("Voltage Solution for all Buses:\n", voltages)

I wasn't able to make too much progress this week due to this being a big exam week. I primarily focused on working on the presentation, but I was unable to attend the meeting on Sunday due to a time conflict I had at the different meeting time.

We had the kickoff presentation this week. My goal was to find a suitable dataset with which to test the current ML pipeline I have. I researched and found a few that I could use including MATPOWER, PGLib, GridSTAGE and build a preprocessing pipeline for the data. I wont be able to do much work next week due to projects and tests. I also have a competition over the weekend. Hopefully I'll be able to get back on track at the beginning of Feb. This leaves for the semester:

• Research the structure of listed datasets to see viability
• Loads datasets into tensors to be read by the network
• Interpret outputs meaningfully
• Update UI accordingly

The tasks that I did this week were:

• Researched publicly available power system datasets suitable for power flow analysis.
• Reviewed MATPOWER test cases, examining their structure and suitability for different load flow methods.
• Investigated Open Power System Data (OPSD) to assess its relevance for HELF-based studies.
• Looked into PGLib-OPF (Power Grid Library) and its benchmark datasets for AC optimal power flow problems.

The tasks that I did this week were:

• Compared the structure of GridSTAGE synthetic grid scenarios with traditional power system datasets.
• Identified key preprocessing steps required to adapt OPSD and MATPOWER datasets for analysis.
• Explored real-world power system datasets provided by utilities and research organizations.
• Investigated whether datasets from ENTSO-E or EIA could provide meaningful test cases.

The tasks that I did this week were:

• Examined the availability of historical load and generation data from open-source repositories.
• Assessed the granularity of different datasets—hourly vs. minute-level data—and their implications for modeling.
• Researched synthetic dataset generation techniques and whether they could supplement existing real-world data.
• Studied case studies that used PGLib-OPF to understand the applicability of its datasets.

The tasks that I did this week were:

• Reviewed dataset licensing and accessibility issues to determine feasibility for research use.
• Investigated possible ways to merge multiple datasets for a more comprehensive power flow test suite.
• Looked into academic papers that referenced large-scale power system datasets and evaluated their sources.
• Compiled a preliminary list of datasets that align with HELF and PINN research goals.

The tasks that I did this week were:

• Refined dataset selection criteria based on completeness, granularity, and compatibility with power flow solvers.
• Evaluated the data cleaning and transformation steps required for each dataset.
• Identified potential challenges in integrating multiple datasets for a unified analysis framework. Currently this leaves the checklist with:

• Research the structure of listed datasets to see viability
• Loads datasets into tensors to be read by the network
• Interpret outputs meaningfully
• Update UI accordingly

The second task on the list should take the most time out of the remaining ones, as updating the UI should be trivial, and the interpretation of the outputs should arise out of the choosing of the dataset.

I had a lot of tests the previous couple of weeks, in addition to spring break, but this week I was able to implement a preprocessing pipeline for matpower testcases using the matpower-pip package. I plan to write a training loop using the 9-bus test case system next week, leaving the only remaining task being to update the UI accordingly and interpret meaningful outputs by April 7th. This leaves us with the following checklist:

• Research the structure of listed datasets to see viability
• Loads datasets into tensors to be read by the network
• Interpret outputs meaningfully
• Update UI accordingly
  This is the preprocessing function:

def preprocess_matpower_case(case_name):
    with Matpower(engine='octave') as m:  # run as context manager
        mpc = m.eval(case_name, verbose=False)
        mpc = m.runpf(mpc)
    # Extract bus and branch information
    bus = mpc.bus
    branch = mpc.branch

    # Extract number of buses
    num_buses = bus.shape[0]

    # Extract node features (e.g., voltage magnitude)
    voltage_magnitudes = torch.tensor(bus[:, 7], dtype=torch.float32).view(-1, 1)  # Column 7 = VM

    # Build edge_index for PyTorch Geometric (bidirectional edges)
    from_bus = branch[:, 0].astype(int) - 1  # Convert 1-based to 0-based
    to_bus = branch[:, 1].astype(int) - 1
    edge_index = torch.tensor(
        np.array([np.concatenate([from_bus, to_bus]),
                  np.concatenate([to_bus, from_bus])]),
        dtype=torch.long
    )

    # Return a PyTorch Geometric Data object
    return voltage_magnitudes, edge_index

I finished building the preprocessing pipeline, including a train/test split, batch sizes, perturbations, etc. It looks like to generate the dataset from testcases will take forever on my machine. Hopefully I will be able to get access to some remote servers to do this. For now, it seems like Im on pace for this semester, as there is at least code which can take in a dataset and train an nn. This leaves our checklist in the same state as before. I added a function to generate data from matpower cases and a function to put the GNNs into a training loop.

I multiprocessed the data generation phase of the training and sent it over to Sam so that we could use the GTRI computers to train an initial version of the network. It is unlikely that this will be done by presentation time. Seeing as this version of the generate_dataset() function was able to run for about 5 samples on my machine it should be able to run in a reasonable time for 500 samples on the compute over at GTRI. If Sam is unable to run it at GTRI, I will apply for PACE access next semester to run it. This leaves the checklist in the following state: For next semester the goals look like:

• Train ML model on PACE
• Validate model locally
• Interpret outputs meaningfully
• integrate model with the rest of the simulation

Sam ran the model on GTRI servers, so all that's left for next semester is analysis and integration.

• Train ML model on PACE
• Validate model locally
• Interpret outputs meaningfully
• integrate model with the rest of the simulation

I focused on organizing results and background material into slides for the upcoming presentation, outlining the motivation, data-generation plan, and initial GCN architecture. I also reviewed the current codebase to confirm what remains before running a first full experiment on the PACE cluster. The immediate goal is to finalize the presentation and prepare for cluster-based training.

I refined the presentation deck, adding preliminary figures and a clear roadmap. I also planned the steps required to launch dataset generation on PACE—such as writing job scripts and arranging storage—though the actual configuration is still pending. Since cluster access is available, my goals for september are as follows:

• test cluster access
• get a preliminary test run on the cluster
• get a full run on the cluster

I didnt make much progress this week since I was preparing for an interview.

I am going through the PACE documentation to figure out how to connect to the cluster. To be thorough, I will document the steps that I used to get onto PACE here:

1. ssh vmodgil3@ login-ice.pace.gatech.edu

2. Since we don't want to run compute-intensive jobs on the login node, we can request a node using slurm: salloc --time=01:00:00 --ntasks=1 We do not specify a node, since pace allocates us one according to our needs. If you want a specific compute node (ie cpu, gpu), you can view this info using sinfo -N -l. This will list nodes with their allocations as well as wheter its a cpu/gpu node.

3. Since the repo is public, you can just directly clone it intoour environment.

I still need to figure out the appropriate way to get a python package manager onto a node. Maybe I should just zip the required packages up and scoop it onto a pace node?

Apparently you need to set a virtual environment up in order to use PACE. I have imported all the proper packages into pace, but it seems like octave is not available. I will need to refactor my script to use matlab and figure out how to use it in order to train on these testcases.

I verified that I could run jobs through Slurm and that my environment loads correctly on allocated compute nodes. I began refactoring the dataset-generation pipeline so it runs cleanly within PACE’s module system. I also created and tested a Python virtual environment with all required dependencies. Octave was successfully installed, so migrating to MATLAB is no longer necessary.

Checklist

• Confirm Slurm access
• Create Python virtual environment
• Validate Octave availability
• Begin GPU planning

I tested the Octave-based MATPOWER components directly on PACE and confirmed that they execute without major issues. I reorganized the file structure so that paths behave consistently between local and cluster runs. I also started looking into what GPU options PACE provides for Octave, since this will matter for scaling.

Checklist

• Test MATPOWER + Octave on compute node
• Fix file path inconsistencies
• Determine GPU library support for Octave

I ensured that the virtual environment activates properly inside both interactive sessions and Slurm batch jobs. I cleaned up parts of the dataset-generation script that relied on local assumptions. I also began checking which parts of the pipeline are most likely to benefit from GPU acceleration, even though I haven’t made any code changes yet.

Checklist

• Validate environment activation under Slurm
• Remove local-only code assumptions
• Identify key bottlenecks

I ran several trial jobs to confirm that Slurm logging, working directories, and output behavior are all consistent. I also read through Octave GPU documentation (primarily OCL) to assess feasibility. At this point, the pipeline runs, but it is still fully CPU-bound.

Checklist

• Validate logging + output behavior
• Review Octave GPU/OCL documentation
• Evaluate if OCL installation is necessary

I reorganized the dataset-generation workflow into clearer steps (loading, running MATPOWER, saving outputs). This should make the GPU-related changes easier later. I also did initial runtime checks to understand expected slowdowns for larger test cases.

Checklist

• Split pipeline into modular stages
• Run timing tests
• Decide how to structure GPU adaptation

I refined the Slurm templates for consistent environment loading, logging, and job reproducibility. I also tested Python→Octave calls through both command-line wrappers and direct invocation. At this point, the whole pipeline works on CPU nodes, but performance will need improvement before scaling.

Checklist

• Refine Slurm job script template
• Test Python→Octave interactions
• Prepare initial GPU-job template

I tested MATPOWER across multiple case files stored on PACE and confirmed that the dataset-generation workflow does not break on different buses/sizes. I verified that all required inputs are accessible from the cluster filesystem. I am now preparing to modify the Octave scripts so they will eventually work on GPU nodes (but I have not made changes yet).

Checklist

• Test various MATPOWER cases
• Verify filesystem access
• Outline which functions need GPU-safe rewrites

I performed another round of full-pipeline tests through Slurm to ensure consistent runtime behavior. I identified which sections of the Octave code cause the largest slowdowns, but I have not yet implemented any vectorization or GPU adaptations. The pipeline is stable and ready for GPU-focused updates.

Checklist

• Full pipeline test
• Identify CPU bottlenecks
• Plan GPU rewrite tasks

I focused on understanding what GPU support Octave actually has on PACE. I checked available modules and experimented with simple GPU commands to gauge compatibility. Most of the work this week was exploratory, since no GPU-safe rewrite has been done yet.

Checklist

• Check Octave GPU module availability
• Test simple GPU commands
• Confirm whether OCL or CUDA backend is feasible

I validated that MATPOWER loads correctly even under longer-running jobs. I tested the dataset-generation flow with larger MATPOWER cases to ensure the pipeline doesn’t break on memory usage. I also started mapping which parts of the Octave code will need modification, but again, no actual rewrites yet.

Checklist

• Test long-running Octave jobs
• Validate large MATPOWER case loading
• Document GPU rewrite requirements

I cleaned up configuration files, standardized job output directories, and ensured the scripts run without manual intervention. I ran small, end-to-end dataset-generation tests to ensure consistency. Everything runs, but remains CPU-only until GPU work is implemented.

Checklist

• Standardize scripts + directories
• Run small end-to-end test
• Begin implementing GPU changes

I verified that MATPOWER executes correctly inside multi-hour Slurm jobs without crashing or hanging. I tightened the job scripts to improve error capture and logging. At this stage, the environment is fully set up — the final remaining major task is rewriting the Octave code so it can take advantage of GPU nodes.

Checklist

• Verify multi-hour job stability
• Improve error/logging structure
• Implement GPU-friendly rewrite