FAQ Data Sources - greenriver/hmis-warehouse GitHub Wiki
Can I edit a csv file before uploading it to the warehouse?
When doing a manual upload, if you open (unzip) the csv files that you downloaded from HMIS and then open the individual files in Excel and save them, you will need to fix the format of the date fields before the warehouse can use the files. Excel will automatically reformat any date field to a format that is incompatible with the HMIS CSV specification, and will prevent successful import into the warehouse.
To format the column(s) containing date-time fields (DateUpdated, DateCreated, etc.), click the column to select it, right click and choose "Format Cells". Click on "Custom" and then enter in "yyyy-mm-dd hh:mm:ss" and then click "OK". Follow the same process for date fields (EntryDate, InformationDate, etc.) which uses a different format “"yyyy-mm-dd”.
Once all dates have been formatted, save the file, rezip the files together, and re-attempt the upload.
NOTE: When zipping CSV files, make sure you are selecting the actual files and not the folder in which the files reside.
How do I check the status of my upload?
Once a zip file has been uploaded, the system will begin processing and looking for possible errors. Updates to client records will not be available immediately in the warehouse. They will be available once an update process runs, which may take several minutes to several hours to complete, depending on data size and current warehouse usage. Some reports that rely on pre-calculated data may not be updated until the next day.
An indication of how many client enrollments remain to update is visible on the Data Source page.
Example of outstanding enrollments to process
If you click on the Uploads View link within a data source, you can see a list of previously uploaded files and their current import status: In Progress, Completed (with a processing time), or Failed. A “completed in X minutes” status means the zip file has been parsed, but does NOT mean there were no errors or all the client records have been updated yet. Full processing of an upload, where client records and reports reflect the new data, may take up to 24 hours.
How do I know if the automated upload is stalled?
Data sources will be marked with a yellow note saying “same file since [date]” when an automated upload is stalled. In addition, a yellow clock icon at the top-right of the warehouse when at least one automated data source has stalled. Clicking on this, takes you directly to the data source page.
An easy way to determine if the file made it to the warehouse is by clicking on the "Automate HMIS CSV Uploads" in the data source. On this page you will see a list of the files received from the automated data source. You can check the top of the list for the most recent file. If there is no recent file, it is recommended that you check with the originating provider.
Once the import starts again, it may take up to two days for the yellow icon and notification to disappear. The two-day period allows the system to confirm the data is flowing.
Example of a stalled upload
Can I delete an uploaded file?
Unfortunately, there is no easy way to delete files that have started the upload process. Once the import starts, data begins to get merged in the system.
Depending on what data was included in the file, uploading a new file with correct information will replace the other uploads.
If the data is from the wrong HMIS or includes data that should not be there then Green River can work through removing it.
When was the last data upload?
Depending on your access and permissions there are few places where you can identify when the last upload occurred. These include:
- Data Sources - Provides the most recent import run for each data source.
- Uploads Table on a Data Source - Provides a comprehensive list of all uploads made to a data source.
- Data Source Report - Displays the Last Import Run without giving the user full-access to the data sources.
Example of Data Source Report