Sherlock drums the tips of his fingers on the table while the waitress fetches them both coffee. He scowls a little at the mention of his absence, which still bothers him, but the face he makes is clearly directed at the ceiling and, by metonymic extension, their captors. "Some sort of memory tampering," he says after a moment. "The things they do with us--well, no matter."

It's far from no matter to him, but it's just clarified his resolve to master and escape this place one way or another. "The city," he gestures, "has it shifted before? In your memory?"