As a kid I spent a few weeks in a "convalescent home" because of my asthma and my mother drove several hours to visit me every week. But most of the other kids rarely, if ever saw their families. Some had been left there as infants and never saw their parents again.
As a young adult I worked at the California School for the Deaf and, while some kids went home every weekend, others only did so a few times a year when the school was closed for vacations. In some cases that was because the parents lived great distances from the school, but others who were closer didn't know Sign Language and couldn't communicate with their own children. Now I understand the school sends all students home every weekend and those who live far away stay with friends.
I knew some Deaf adults who grew up in a state hospital for "mentally retarded" when their intelligence was normal. They had even developed their own sign language in order to communicate with each other. Today they would never have been sent to an institution like that. And even the patients who are correctly placed in developmental centers usually have opportunities to go out into nearby communities if they can handle doing so.
Today people with special needs are likely to be seen anywhere and that's helping to overcome some of the stigma attached to disabilities. Things have certainly improved.