Since reading the input still has to go through the I2C expander, it would also have the 2ms limit. There are two pins, IO2 and IO3, which can be faster (https://communities.intel.com/message/207904#207904) - 477kHz to 2.93MHz.
You might be able to use these two lines to bit-bang a protocol to an external ADC if you get one that uses I2C since SPI would require a third line.
I ran into the same issue and ended up using a MSP430G2553 to handle high-speed behavior (200ksps internal ADC) then buffering the results before sending it back to the Galileo.