Use built in readers with multiple inputs - wolfma61/CNTK GitHub Wiki

See the description at Understanding and Extending Readers and look for the section describing how to "compose several data deserializers"