I have faced such issues earlier. I can tell that you can try to set your debugger to break at a point exactly where your exception is thrown at a beginning. By doing this a key which was not present earlier will be accessible to you as the value in your call stack.
If you are using the Visual Studio then in your Visual Studio you can find this option as shown below :
Debug → Exceptions... → Common Language Runtime Exceptions → System.Collections.Generic
At this path must check a Thrown check box.
If you want to address more specific instances where the information is needed by you at runtime then you need to provide your code with the use of
IDictionary<TKey, TValue> and it will not be tied directly to your
Dictionary<TKey, TValue> also you can try to implement your own dictionary class which will provide you the required behavior.