The 94-year-old's health deteriorated in the middle of the night and he was transferred from his home in Houghton, Johannesburg, to a hospital in Pretoria.
President Jacob Zuma's spokesman Mac Maharaj told South Africa's Enca Television that Mr Mandela's condition was "serious this time".
He said Mr Mandela was receiving expert medical care and doctors were doing "everything possible to make him better and comfortable".
"President Jacob Zuma, on behalf of government and the nation, wishes Madiba a speedy recovery," he added.
It is the second time in recent months Mr Mandela, a revered figure across the globe, has been in hospital - and the fourth health scare in six months for the anti-apartheid leader.
The last time was in May/April earlier this year when he was kept in hospital for 10 days to receive treatment for pneumonia.
Mr Maharaj told Sky News Mr Mandela was conscious, was "able to breathe on his own" and that he was "communicating" with relatives.
"Some members of his family have been there with him all the time," he said.
"It is a condition he has had over a long period of time ... and we hope this time it will also be treated successfully.
"He is a fighter. He has been through this many times and he has been through worse issues, and he's survived.
"For all of us who are concerned, I understand that, our admiration, our hopes for him - we'd like him to be with us forever.
"But we also know we are all human beings, our life is transient, and therefore, with our prayers, with our thoughts, I know we will help him to be strong."
Mr Mandela's wife cancelled an appearance at a hunger summit in London later.
It is thought Graca Machel, who was due to address the Nutrition for Growth meeting, has flown back to South Africa to be with her stricken husband.
ANC spokesman Jackson Mthembu said the nation was "prepared for the worst", but urged people to pray for the icon.
He told Sky News: "Madiba is the personification of humanity, friendship, honesty, integrity. He means all of these things to us.
"He is also the person who without any hesitation took the apartheid state head on and suffered for that, but came out of prison undeterred. He held the hand of his oppressor, held the hand of his jailer and said, 'can we all reconcile and build a better South Africa?'. There is no better person.
"Without the calibre of Madiba's leadership, the likelihood is that South Africa would have gone to ruins ... but there was a Madiba.
"He not only spoke to everyone, but spoke reason to everyone. That will be the legacy of Madiba."
Speaking from Pretoria, Sky News Special Correspondent Alex Crawford said South Africans were "very concerned" about Mr Mandela's ailing health.
"A 94-year-old man, of frail health, who has already been several times in hospital over the last couple of months - having to be taken into the hospital at 1.30 in the morning, local time, is not good news," she said.
"That means he must have been in a particularly serious condition that they couldn't deal with it at his home, where they have 24-hour medical care available to him, provided by the top military doctors.
"In that home he has emergency equipment, resuscitation equipment, he has everything that is necessary, needed and what you'd expect to look after a former president - but that wasn't enough.
"Clearly his situation deteriorated to such an extent that they felt he needed to be moved immediately to a hospital, and I think that says a lot."
She continued: "He is due to be 95 on July 18, he is a good, old age, and he has suffered terrible health, particularly in the past few months. Each time one imagines a stay in a hospital really does take its toll, particularly on an old man.
"Last time, shortly after he was released from hospital, President Zuma visited him at his home with members of the ANC hierarchy and those were the last pictures we saw of Nelson Mandela in public.
"There are a lot of good wishes coming from the general public, but also an appeal ... to let him be, to let him recover in peace.
"Whilst he's still in hospital and he's still considered to be in a serious condition, the nation will be holding its breath once again."
Mr Mandela has a history of lung problems dating from when he contracted tuberculosis as a political prisoner.
He spent 27 years on Robben Island and in other jails for his attempts to overthrow the white-minority government.
The Nobel Peace Prize Laureate, who became South Africa's first black President in 1994, is a global symbol of tolerance and the struggle for equality.
He stepped down as president in 1999 and has not been politically active for a decade.
