It is all depends on the organization. Most of the big companies they keep the backups externally. For an example there are lots of backup software available and one of the popular one is veritas backup. Only small companies they keep the backup in their second hard drive. Big companies, but their backup in tape drive or in SAN storage.
There many ways to take and keep backup safe. I would try to highlight few of them.
1 Onsite backup: for this backup is taken in the same datacenter area. it can be on tapes or it can be on de-duplication devices or backup storage. If backup is taken on tapes. Tapes shall be kept in fireproof closets.
2 Offsite backup: storage location is different than datacenter. If tapes are being used, then tapes are shipped to offsite location as per routine plan. in case of de-duplication devices, Offsite backup device, sync with onsite backup device on daily basis or as scheduled.
3 DR location backup: in this backup is done online and replicated on DR location servers. It also depends on applications.
As stated by remier, it completely depends upon the company. If they want to secure the data from all the possible disasters, they will mostly opt for offsite backup solutions. In offsite backups the data is stored in a different data center far from the data center where you website is hosted. Software's like R1Soft CDP solution can offer you full data protection in the event of data loss. Using such remote solutions you are able to restore your data anytime you need. This R1Soft software offers features like Differential backups and Incremental Backups along with End-to-End Strong Encryption to secure your data.
I have always kept our backups in a completely different location, in another datacenter. It is, IMO, the safest option, as if something happens in one location, the backups remain unaffected and can be easily restored.
However, I've heard of others who feel comfortable keeping it in the same location, so it is whatever you are comfortable with.