How to recognize RER 3(which go to Versaille palace) at the station, i do not want to mix it up with another train? Thank you

The RER C trains going to Versailles Palace would list "Versailles Chateau Rive Gauche" as their final destination / terminus on screens listing upcoming trains.  This detination name (and others) will be shown on the overhead TV screens above the RER C train platforms you would be standing at waiting for the RER C train to arrive for boarding.  And this destination would also be shown at RER C stations where there are multiple platforms serving several RER C trains going to different destinations
The name may be abbreviated / shortened ("Versailles Chateau RG" or "Versailles RG", but those are the words you want to see: "Versailles" and "Rive Gauche" or "RG".  If you don't see those words/letters, or you see other "Versailles" destations like "Versailles Chantiers" the next RER C train is going elsewhere and you should wait for the next train and keep checking the overhead screens on the RER C platform, until you see "Versailles" "Rive Gauche".  If you don't see that final detination name, you may be standing at the wrong platform.
 
See this article on RER Trains to Versailles to see an example of the overhead screens displaying the destination of the train you want (in the linked example it's "Versailles Chateau RG" on the right hand screen.  That line will show at time the Versailles Chateau train will arrive at this station, the 4-letter code that will be shown on the front of the lead car of the train ("VICK" in this photo) and from which platform the train will be arriving/departing from (platform A, in this case).