I have two clients using matchmaking via the session interface and I'm getting a found result (yay). Next question is which of the two clients should start the listen server? Our game is standard Unreal client-server and one of them needs to be the server...
So it turns out that UE4 sessions have a bHosting flag that is supposed to be used for this, but Oculus doesn't set it. Instead I found that whoever matchmaking assigns as the creator of the session has their Oculus name in the session's owner field. I'm checking that against the local oculus player name to determine who is server and who is client.