Add test for retry out of the retry_window.