Use built in readers with multiple inputs - eldakms/CNTK GitHub Wiki

See the description at Understanding and Extending Readers and look for the section describing how to "compose several data deserializers"