After which: Headcanon based on SPOILER FOR END OF "INFINITY WAR"
When half the Earth's population abruptly disintegrated, that tragically included Clint's wife and sons. He tried to flee with his daughter who then also turned to ash in his arms. The combination of her ashes and Barton's fists hitting the ground seemed to collapse it beneath him, crumbling away as the people had, and he blacked out with grief and fury with a last sensation of falling. He woke up here. What's keeping him basically on his feet and coping is the suspicion nothing is real and he's had a psychotic break, which he might as well make the best of, in the hopes when he comes out of it, his wife and kids will be fine. —Plus actually coming in and out of a psychotic break: he also spent so much time away from them, dissociation/confusion—imagining himself in different times (perhaps before he married Laura or the kids were born thus forgetting them entirely)—is horrifically easy. He'll alternate between his usual self and falling through the fissures. This is true even before consciously remembering the Dusting; the memory is gone but the trauma remains.
In short: often he won't remember his wife and kids, even though they did exist, due to trauma from their Dusting.