Your website says the ticket office at Orly that sells Navigo Découverte cards is only open until 18:00.
Does this mean that if we arrive at Orly at 21:10 there is nowhere at Orly to get the Navigo Découverte card?
Yes, unfortunately the ticket window for Navigo pass sales at Orly airport (Terminal 4 bus loop) is closed at 6pm / 18:00. This is the only place to buy Navigo at Orly airport as far as I know.
The cheapest public transit for Orly Airport to Paris is Tram 7 + Metro 7. Tram 7 departs Orly Airport from Terminal 4 and goes to Villejuif Louis Aragon & costs one Paris Metro/Tram/Bus ticket (a "ticket t+", costing 2.10€ in 2023). Then you'll need one more Metro ticket for Metro line 7 train to get from Villejuif to Chatelet station, at which I would recommend you try to buy Navigo passes. This is the largest most central Metro/RER station in Paris and is most likely to have Navigo passes available at RER or Metro ticket windows.