Steven ellis notebook - snowpuppy/augreality GitHub Wiki
Meetings
- 4/24/2013 (2 hours): First meeting
- 5/9/2013 (2 hours): Second meeting
- 6/23/2013 (2 hours): Third meeting
- 7/7/2013 (2 hours): Fourth meeting
- 7/21/2013 (2 hours): Fifth meeting
- 8/4/2013 (1 hour): Sixth meeting
- 8/18/2013 (2 hours): Seventh meeting
Started learning blender, created a few simple 3D models.
Meetings
- 8/22/2013 (1 hr) Meeting to work on Project Proposal
- 8/23/2013 (2 hr) Meeting to discuss interface for CCU and Headsets.
- 8/24/2013 (4 hr) Meeting to test XBee devices, set up notebooks, and discuss CCU/Headset interface.
Preliminary Proposal. Notebooks created.
Weekly Work Total: 7 hours
Project Work Total: 20 hours
Individual Meetings
- 8/29/2013 (1 hour): HW1 Final Project Proposal completed
Final Proposal.
Weekly Work Total: 5 hours Project Work Total: 25 hours
Meetings
- 9/1/2013 (2 hours): Meeting to test range of XBees using elevated platform
- 9/2/2013 (2 hours): Work on Project Specific Success Criteria, describe requirements for preliminary headset schematic
- 9/4/2013 (1 hour): Edited Project Specific Success Criteria
- 9/7/2013 (3 hours): Comparison range testing with larger antennas and 2.4 GHz modules, HW2 PCB Design completed
- 9/8/2013 (2 hours): Started reading OpenGL red book/looking through ES2.0 example code
Saturday meeting. Initial OpenGL research.
Weekly Work Total: 10 hours Project Work Total: 35 hours
Meetings
- 9/12/2013 (1 hour): Tested power consumption of Raspberry PI Model A and composite screen
- 9/13/2013 (2 hours): Disassembled LCD frame and identified various components on the board.
- 9/14/2013 (4 hours): Continued screen power testing, start I2C and IMU drivers
Got pi model A and mystery composite LCD. Identified components on mystery composite LCD board. Did measurements of power consumption on model A and mystery LCD. Model A running openGL demo: ~300mA. LCD: works with supply voltage of 6v-12v, with current usage ranging from 130mA-260mA depending on input voltage. Plan to power LCD is to snip 5V regulator and directly feed in 5V.
Weekly Work Total: 7 hours Project Work Total: 42 hours
Meetings
- 9/15/2013 (4 hours): Met with Thor to build the initial headset prototype. A baseball cap had a large slot cut into the bill where we mounted an enclosure with our LCD mounted inside and the reflector glass taped on. Plugged into the pi, we were indeed able to see the display as a 'floating' image. However, due to the short distance from the eyes, focusing on the display and background simultaneously is impossible.
- 9/17/2013 (2 hours): Created presentation for packaging TCSP, started work on report.
- 9/18/2013 (2 hours): Measured weights of currently available components for packaging report, continued work on report.
- 9/19/2013 (2 hours): Finished written portion of packaging report, created CAD drawings.
- 9/20/2013 (1 hour): Started work on writing the rendering engine, by working on C++ vector math class.
- 9/21/2013 (4 hours): More work on packaging prototype, discussion of data formats.
Packaging TCSP/report. Created prototype with baseball cap. Initial work on graphics code (vector classes).
Weekly Work Total: 15 hours Project Work Total: 57 hours
Meetings
- 9/27/2013 (2 hours): I talked to Blaine and asked him some questions about the Incredible HUD. Some highlights: for packaging, a motorcycle helmet was used with a custom cut sheet of acrylic with reflective automotive window tint applied instead of a half-silvered mirror. They used a laser projector with the hopes of adding an optical collimator to improve focusing, but weren't able to find a suitable collimator to buy.
- 9/28/2013 (4 hours): Went with Thor to Hobby Lobby and Menard's. We looked at various possible headset mounting solutions: a welding mask, hard hat, and face shield. The welding mask seemed good for mounting things to but also restricts your field of view. This could be helped by cutting out a larger viewing hole, though. The hard hat was lightweight and cheap, but probably somewhat harder to mount to. We ended up buying a hard hat, some nuts/bolts, angle aluminum for mounting, and a sheet of acrylic to potentially use for our display reflector. We visited Autozone to see if they had any reflective window tint but they didn't.
Accomplishments
Some research was done into the optics required for our display. Materials were obtained for working on the packaging prototype.
Weekly Work Total: 6 hours Project Work Total: 63 hours
- 10/5/2013 (4 hours): Met with Thor to work on construction of the second prototype, using the hard hat. The basic approach of this prototype is to have a frame of aluminum angle bars mounted to the hard hat to which the display and electronics enclosures are then mounted. Holes were drilled into the side of the hard hat for mounting and two pieces of angle aluminum were cut to appropriate lengths and bolted into the holes. Pictured is the partially constructed prototype with the display enclosure and cross bars sitting on top for future assembly reference.
- 10/6/2013 (5 hours): Met with Thor to continue work on constructing the prototype. The aluminum rails had slots cut into them and two new cross bars were cut such that when bolted together it would form a 'cradle' in which the display enclosure would sit. The bars were placed accurately enough that the display enclosure is fairly secure with just a friction fit in between them. Then the precise dimensions of the display were measured and a hole cut into the enclosure on top of which the display would be place. The display is temporarily taped on top of the enclosure and the enclosure slotted in to the aluminum frame. This leaves two main components left for the packaging prototype: the reflector sheet and the electronics enclosure. The current setup is fairly front heavy (much more so than the baseball cap prototype because it sticks out a greater distance and the frame adds a nontrivial amount of weight) but the addition of the electronics in back, especially the battery, should offset that somewhat. The acrylic sheet we plan on using for the reflective surface is fairly light so should not be problematic.
Accomplishments
Significant progress was made on building the packaging prototype. All that's left is adding the display reflector and electronics packaging.
Weekly Work Total: 9 hours Project Work Total: 72 hours
- 10/12/2013 (3 hr): Work on packaging continued. The brim of the hard hat, which was obstructing the view of the display, was removed with the help of a hacksaw. The PCB/Raspberry Pi/Battery/Xbee stack dimensions were measured to figure out the size of enclosure needed. It was concluded that due to the height of the composite video jack on the Raspberry Pi (the highest component on the model A) and the ethernet jack (the highest component on the model B) that a pin header with more clearance than the ones available in lab would be needed.
Packaging work continued.
Weekly Work Total: 3 hours Project Work Total: 75 hours
- 10/16/2013 (4 hours): Further work occurred on packaging. The reflective film on the acrylic sheet was reapplied to reduce the number of bubbles and improve optical quality. The application method used this time stretched the film slightly resulting in a slight banding effect, but this was not very noticeable in actual use.
- 10/17/2013 (4 hours): More packaging and optics work occurred. A large fresnel lens was cut to an appropriate size and placed in the bottom of the display enclosure. This substantially increased the field of view of the display. Also it slightly increases the apparent focusing distance from the user's eyes, but not by much. The virtual image distance looking through the lens is calculated by the formula 1/((1/distance)-(1/focal length)). In our case, with ~2" distance between the lens and display and ~12" focal length of the lens, this works out to a 2.4" virtual image distance. This could be substantially improved by a combination of using a slightly farther distance between lens and display and purchasing a lens with a much shorter focal length. With a 2.4" focal length we would get a virtual distance of 12", which should provide acceptable focusing performance. Unfortunately, large lenses with such short focal lengths are generally somewhat hard to find, expensive, and suffer from high distortion. The shortest focal length that has been found on a reasonably large and low-priced lens was 120mm/4.7": http://www.3dlens.com/shop/pocket-fresnel-magnifier.php. With the current 2" screen distance, that gives a virtual image distance of 3.5". But if the lens to screen distance were increased to 3", the virtual image would be 13" away which would be a very good improvement for only an extra inch of size increase. An OpenGL shader program can be used to cancel out the distortion by adding pincushion distortion to the image on the screen to cancel out the barrel distortion of the lens. This will however require the use of GLES2.0 instead of 1.1, since 1.1 only supports the fixed graphics pipeline functionality and 2.0 moves to the programmable shader model which is in general substantially more development effort in exchange for greater flexibility.
Accomplishments
The optics were improved with a higher quality reflective surface and the addition of a fresnel lens which substantially increased the field of view. Additionally some work was done on graphics.
Weekly Work Total: 11 hours Project Work Total: 86 hours
- 10/23/2013 (3 hours): Work was done on the GLES2 code. Primarily setting up a basic framework of needed helper functions, like shader loading and window initialization. Found a model loader library called tinyobjloader and wrote some code using this library to load data from a wavefront model file into an opengl vertex buffer.
- 10/26/2013 (3 hours): A meeting occurred with Alec and Stephen. A SPI master 'hello world' demo was created on the Raspberry Pi and the correct output was observed on the oscilloscope.
Scope waveform:
]
Accomplishments
Progress has been made on the rendering code. SPI communication between the STM32F4 and Raspberry Pi is now working.
Weekly Work Total: 6 hours Project Work Total: 92 hours
- 10/31/2013 (3 hours): Once the board was completely assembled and attached to the support boards (raspberry pi, xbee, IMU, and GPS) with appropriate standoffs, it was possible to make measurements to design the
- 11/1/2013 (5 hours): The GPS was tested along with Stephen and Thor by taking a walk outside around the engineering fountain and logging the data to a file. Then a graphics demo was created which displayed the object moving along a path described by the GPS data. Here is a video of the demo. This worked reasonably well, but there was very noticable stuttering when the object was fairly close to the user/camera position. This may improve somewhat with more favorable weather conditions where the GPS would be able to connect to a greater number of satellites.
- 11/2/2013 (5 hours): The CAD model of the rear enclosure was finalized. An attempt was made to use the 3D printer in the IEEE office to print the enclosure, but due to a problem with the model file exported by sketchup the print did not succeed. So for the interim, a cardboard box was constructed to the dimensions of the CAD model. The final aluminum rail was screwed onto the headset assembly. A female header was soldered to the display power cable so that it can be powered by the supply on the custom PCB. The rear enclosure and cables for display power and the GPS antenna were all taped to the headset. With all of this done, the headset is finally a fully mobile prototype, being powered solely by the battery.
Accomplishments
Rendering based on GPS data has been achieved. A packaging prototype has been finalized.
Weekly Work Total: 13 hours Project Work Total: 105 hours
- 11/5/2013 (2 hours): Autodesk Inventor was downloaded and used to recreate the 3D model for the rear enclosure. Another attempt was made at printing the enclosure using the IEEE 3D printer. This time the model file was loaded correctly with no holes in the bottom. The print started out quite well, but partway through the part warped enough that the print could no longer complete.
- 11/6/2013 (1 hour): A brief meeting occurred with Thor and Nick Molo, who is largely responsible for the maintenence of the IEEE 3D printer. The failure of the previous print was discussed and some suggestions were made by Nick on how to avoid the problems. He recommended printing without creating a raft and with a higher bed temperature than was being used (120C instead of 110C). The model file was modified and re-exported with these changes. This new file was emailed to Thor, who planned to meet with Nick the next day to print the enclosure.
- 11/8/2013 (2 hours): A meeting occurred with Stephen. The printed rear enclosure had completed, but the bottom was severely warped. Aside from this problem, several other flaws in the design were noted. The battery compartment was slightly too small, preventing the battery from seating properly. Similarly, the slot for the composite video jack was not quite large enough. Also the GPS antenna which had been planned to go through the composite video slot was found to have too stiff of a cable for this without widening the slot. The hole for the XBee antenna was not aligned quite properly. Aside from this, the other ports were all appropriately aligned and sized. Working with Stephen, a small amount of plastic was trimmed from the aforementioned locations to make everything fit together better. With these small modifications, the PCB stack and battery fit in well enough for an acrylic lid to be secured with tape. Later in the day, a power switch for the headset and a 4"x6"x2" ABS enclosure for the CCU were purchased at Radio Shack. Holes were drilled in it for the XBee antenna and the keypad, display, and power cables. A second LCD was modified for 5V operation, with the regulator chopped off, input and output shorted, and the output inductor shorted. It was verified that the display was indeed operational at 5V by plugging into the custom PCB 5V rail. The LCD and keypad were then mounted to the lid of the enclosure. It was noted that while there is plenty of room in the enclosure for the Raspberry Pi and XBee, the large clearance required by things like the SD slot, composite video jack, and USB jacks actually makes the inside of this enclosure very tightly packed.
This is the printed enclosure.
Accomplishments
A 3D printed enclosure has been added to the headset, replacing the original cardboard box. Additionally a CCU enclosure has been constructed.
Weekly Work Total: 5 hours Project Work Total: 110 hours
- 11/10/2013 (5 hours): A meeting occurred with Stephen and Thor. Extensive work on packaging occurred. The front enclosure was cleaned up substantially and much of the wiring and the GPS antenna were moved underneath the helmet.
- 11/11/2013 (1 hour): The excess length of aluminum rail extending past the front enclosure was cut off.
- 11/13/2013 (8 hours): A meeting occurred with Thor. The SPI protocol for sensor data was worked out. The protocol decided upon was: the Raspberry Pi sends a 1 to the microcontroller then waits for the micro to respond with the size of the packet in bytes, which is 22. Once it received the size, the next 20 bytes were packed 32 bit floats in the order of: GPS latitude, GPS longitude, IMU pitch, IMU roll, IMU yaw; followed by another two bytes holding the RSSI and battery charge level. GPS data was successfully sent, but IMU was not yet integrated. GPS data was also integrated into the cube rendering demo.
- 11/16/13 (8 hours): The rendering code was substantially updated, now with a model loader and the ability to rotate the camera about all three axes based on IMU data. A meeting occurred with Carlson, Thor, and Alec. In this meeting the IMU data which Thor had gotten sending over SPI the previous day was integrated with the rendering code to rotate the camera.
Packaging was further refined. Model loading was added to the rendering code. IMU data was integrated into both the SPI and rendering code.
Weekly Work Total: 22 hours Project Work Total: 132 hours
- 11/17/13 (4 hours): A meeting occurred with Thor and Alec. It was realized that the camera axes only worked correctly when the headset was facing north. The pitch and roll axes started behaving very strangely when the headset was in other orientations. Much time was spent trying to figure out the correct orientation for the camera. After coming home from lab the realization was developed that the problem was most likely due to the order of the camera rotations. With the yaw occurring last, this made the pitch and roll rotations occur about a different axis than the user would be expecting. Thus, reasonable behavior happened when the user faced north and the camera acted in bizarre ways anywhere else.
- 11/18/13 (2 hours): The order of the camera rotations was changed from pitch, roll, then yaw, to yaw, roll, then pitch. After this very simple change the camera problem that had caused so much frustration was finally eradicated. A meeting occurred with Thor. The GPS coordinates were being applied in an incorrect orientation in the rendering code, so testing was conduced with Thor to figure how to rectify this. This was not figured out and remains a pending bug.
- 11/19/13 (6 hours): A meeting occurred with Carlson and Thor. The new PCB was causing some troubles which were eventually traced down to improperly configured firmware on the XBee and GPS modules. The Raspberry Pi's SD slot also had one of the contact pins damaged resulting in very unreliable operation.
- 11/21/13 (7 hours): A meeting occurred with Carlson and Thor. The GPS orientation was finally rectified and now works properly.
Rendering based upon IMU and GPS data is now fully functional with all axes oriented properly.
Weekly Work Total: 19 hours Project Work Total: 151 hours
- This week was Thanksgiving break so no work on the project occurred.
Nothing occurred over break.
Weekly Work Total: 0 hours Project Work Total: 151 hours
- 12/03/13 (8 hours): A meeting occurred with Alec and Thor. The rendering code was modified to add in the capability to load a scene from a configuration file. A GameObject class was added which just encapsulates the model mesh, location, orientation, scaling factor, and a visibility flag. An array of these GameObjects is used to store the entire scene to draw. A routine was written to read in a file in a custom text format describing a 3D scene to be rendered. The format of the text file had each object split across two lines, the first of which contained just the array index in which to place the object and the second of which was: (3D/2D flag) (X) (Y) (Z) (Pitch) (Yaw) (Roll) (filename) (visibility flag) (scaling factor). This file is loaded in during the initialization routine, and then the rendering loop was modified to loop through each object in the GameObject array and draw the object in the correct location if the visibility flag is set. Then a C program was written in order to generate this configuration file from a more easily written and human readable format. This format was chosen to be a text file which represents walls as *, pellets as o, and ghosts as g. Empty areas are represented with spaces. The C program just reads in the characters from this file and generates the appropriate line for the application configuration file which puts the correct object type in the correct location. The initial example created is a Pacman-style maze with blue walls and yellow pellets and pink ghosts inside. Unfortunately the current implementation has a fairly low framerate.
- 12/05/13 (4 hours): Some work was done to optimize the rendering code, mainly in the level loading code. Instead of reloading model files for every instance of an object, it will only read each model file once and each object simply contains a reference to the global copy. I had hypothesized that this may have been part of the reason behind the framerate problem, but it really didn't help very much. It did however substantially reduce startup time which was nice.
The headset application can now load a scene from a level configuration file.
Weekly Work Total: 12 hours Project Work Total: 163 hours
- 12/08/13 (5 hours): A meeting occurred with Thor to work on integrating the XBee packets from the CCU into the simulation. Thor added the ability to send a new configuration file to load during runtime, and pause and resume the simulation code. None of this functionality was completed though. Some code cleanup and documentation improvement was done in preparation for the final submission.
Code was finalized for submission.
Weekly Work Total: 5 hours Project Work Total: 168 hours