A Terabyte of Storage Space: How Much is Too Much?

Over stuffed suburban garageI read an article a couple of weeks ago about Microsoft raising the Office 365 storage limit to one terabyte. Office 365 is a solution where the end user pays a monthly fee for the MS Office suite along with hosted storage on OneDrive. I really wonder how much storage is enough? Can I really generate and save enough Word, PowerPoint, or Excel files to fill one terabyte? I decided to dig into it further to see just what will fill 1,000 gigabytes.

Office Documents

It is estimated that 85,899,345 pages of Word documents would fill one terabyte. Now, if you can truly generate that much content, you have a serious archiving task on your hands. I am currently reading the book “John Adams” by David McCullough; it is approximately 650 pages long which means I could archive about 132,152 books of similar size. It has taken me nearly two months to read this book so it would take 66,076 months or 5,506 years to read my entire library. I cannot read that fast, nor do I have that much life left in me.

Music Files

Assuming that an average song takes up five megabytes, one terabyte could fit approximately 200,000 songs or 17,000 hours of music. How many songs do you have on your iTunes right now?

Movies

You could fit approximately 500 hours worth of movies on one terabyte. Assuming each movie is roughly 120 minutes long, that would be about 250 movies. I do know people who have that many movies in their library, so it is possible that they could build a database of movies to fill that space.

Pictures

You could fit approximately 310,000 photos in one terabyte. You could fit even more if you used a compression algorithm. How would you even catalog that many photos? By time, by subject, by category? Suddenly, we are facing big data issues in our personal lives, and we are going to need similar tools to be able to make sense of all of our potential data stores. With digital photography, it is possible to take a lot of photos without ever having to worry about development costs, so maybe 300,000 pictures is not out of the question.

Thoughts

With advances in technology, we have a lot of potential storage space available to us. Microsoft struck the opening salvo, but I expect Google Drive, Dropbox, Box, and others to follow suit. One terabyte hard drives are not uncommon right now and even though we have the potential filespace, can we fill it responsibly? If we can fill it, do we have the skills and tools necessary to keep track of our digital belongings? Perhaps there is a new IT job category—personal archivist—created for those individuals drowning in their digital “stuff.” My point is that we need to take a step back and assess the data that we are keeping and ask ourselves: “Just because I can keep it all, do I need to? Do I have the skills and tools necessary to ever find what I am looking for?”

If we don’t need it and we can’t manage it, maybe it is time to clean out the digital garage. Do you need to clean out your digital garage? Let me know your thoughts.

Author Kelly BrownAbout Kelly Brown

Kelly Brown is an IT professional, adjunct faculty for the University of Oregon, and academic director of the UO Applied Information Management Master’s Degree Program. He writes about IT and business topics that keep him up at night.

Print Friendly, PDF & Email

80 thoughts on “A Terabyte of Storage Space: How Much is Too Much?

    1. sam

      I have 2,157 movies in 1080p (some are in 4k but not many) and 220 TV shows ranging from 720p-1080p.
      Currently using less than 18TB of space across a server, NAS, and external. My “permanent” solution will be 5x12TB HDD’s in RAID 5 for storage and 5x12TB HDD’s in raid 5 on a synology for backups.

      1. Matt

        It starts to become an issue of how much do you really need “online” at any given time. Of those 2,157 movies, how many of them did you watch in the last year?

        How many of them can be found online at any given time on an online streaming website?

        Can you just store most of them on a handful of external hard drives and have an online catalog of each drive and plug them in as needed?

        In the old days (15 years ago). Tv series dvds didn’t exist, movies tended to be $10 and up, and many things just hard to find.

        Now things are cheap, readily accessible.

        And photos. People used to take 1 or 2 good photos at an event like a picnic. Now they take 20 – 50. But instead of deciding which ones to keep, they keep them all. Photos have become the boring home videos of the 80s-90s. How many events will someone care about years in the future. I have maybe 1,000 photos from when i was born to age 18. Someone who turns 18 in 2030 will probably have over 100,000. Even at 5 seconds a picture, it will take someone 8hrs a day, 17 days just to look at all of them. The value of the pics will drop and become something that is always hauled around, but rarely looked at.

        1. David Bernazani

          Matt- so true. Each time I take a photo with my iPhone, I don’t take just one, I take 3 or 4. But, unlike some who may just keep them all, I am constantly deleting most of them except the best ones. I have maybe 200 or so documenting the last 2 years of my life, and still haven’t bothered to upload them somewhere, because I have iCloud backup. I don’t pay anything for extra “cloud” storage, because I don’t need it.
          And unfortunately, storage technology is getting beyond my understanding. I have no idea what “RAID 5” is, and some of the other stuff Sam wrote about.
          As for streaming: I’m starting to stream some shows and movies, but right now I only subscribe to streaming Netflix and CuriosityStream. I know there are others, but who has time to watch them all, or the increasing money it takes to pay monthly subscriptions for ea h one? Oh, and I also still get some Netflix dvd’s by mail, because for some mysterious reason, many of the latest movies are not available to stream on their site. In fact, I’ve done searches of an assortment of movies, both new and old, that I’d like to see (from Casablanca to Back to the Future to Gladiator) and couldn’t find a single one! I had always thought sites like Netflix had gargantuan libraries of most any movie you could think of to watch, but it’s actually quite limited. Anyway it makes me glad that I have my own little video library of all my faves.

          PS Did anyone notice that the title to this thread, “A Terabyte of Storage Space” is already sounding kinda quaint? Sorry, Kelly! Bit look how fast things change in this field!

          1. Kelly Brown Post author

            OK David and Matt, I concede that things are moving much faster than I thought they would. Perhaps I should have been more forward thinking and named the post “A Petabyte of Storage Space…” or “An Exabyte of Storage Space…” :-). I agree with you that we probably don’t need everything that is currently stored and can stand to do a lot of housecleaning. I think about cloud storage like the physical storage units that dot the landscape of America. There are some legitimate things behind those roll-up doors but there is also a lot of stuff that we just can’t seem to part with and continue to pay for month after month. I wonder if we will have a television program in the future called “Digital Storage Wars” where we auction off unclaimed cloud storage to the highest bidder to see what treasures lurk inside :-).

        2. Teri

          My question for you, Matt. How many hours of TV do you watch per year? Just 3 hours a day is equal to 2 movies a day. I rewatch my collection all the time, I watch at least one ‘saved’ movie a day and one new one (not owned, catch it on streaming, or buy it at the pawn shop for $1.00) So I re-watch 365 movies in a year that I’ve already seen. Yes, I’ve accidently memorized the lines from “Notting Hill.” But you are not understanding movie buffs. If I want to watch “Session 9” or “Alive” then THAT is what I want. Try finding those on any streaming service. The trouble with streaming services is that you’d have to have membership in ALL of them (and pay for it) and you still wouldn’t find the exact movie you WANT to watch when you want to watch it. And netflix? Really? Have you seen their selection? This grandma is a scary movie buff and netflix has very little of those. In this world of immediate gratification, the one pleasure I do demand is the movie I want when I want to watch it. My collection allows me that pleasure.

      2. Teri

        Sam,
        1TB is a joke for a movie buff. Consider that watching only 2 movies a day is 730 movies. I do that easily. I too own over 2000 movies — and I don’t keep all that I watch, many are a GFOW (good for one watch) and wouldn’t want to see it again. I’m 58 and a 2000 movie collection is not a huge amount to have saved over the years!

      3. Mrs

        I’m in the market for a new computer the different between a 1TB computer and a to terrabyte computer it’s $300 is it worth to pay the difference?

  1. marlon ferguson

    I have a considerable collection of photos (20k) or so and 13000 and growing in music library , my problem is that i have 300 dvds or so that i purchased most of which I wish to save, I spent alot of time and money over protecting these (300 disc dvd changer pioneer) and would like to preserve, I keep researching media servers for my self, but if i was going to travel cant really take with me, and if i was to convert them all digital and throw dvds away, would like to have a multiple bay type storage and a cloud option…..any thoughts on my conglomerated media issue…..
    use google drive 15 gig
    use apple idrive 200 gig
    have onedrive for business 1000 gig

    none of which work for my complete issue ????

    1. Kelly Brown Post author

      Hello Marlon,
      Thanks for reading our blog. In a blog coming up next week I do a short review of a product by Western Digital called Personal Cloud or a sister product called Personal Cloud Mirror which has two mirrored drives at level 1 RAID. They are meant to be a personal cloud so that you can access your data, photos, music or movies anywhere. The mirroring is important to save your data in the event of a disaster or drive failure. Check it out at: http://www.wdc.com/en/products/personalcloud/ and let me know what you think.

    2. eddie contreras, mpa, ms in IT Quinnipiac U.

      Two things. First it stands to reason Microsoft’s model gives us the impression that things are free or cheap – this branding strategy is important, because IT students and techies like to scavenge for deals to by tools – when these tools are branded bundles then they are perceived as good buys – Microsoft seems to make us think of it as a one stop shop for everything which is arguably a good marketing idea – my second impression Professor is that people commoditize things as such they seem more valuable with more stuff –
      do you agree?

      1. Kelly Brown Post author

        Yes, the Microsoft and other models do make components seem cheap or free because of bundling. For example Microsoft gives you a “free” 1Tb cloud storage with their service but my contention is that you may or may not need that much. If you try to fill it with documents and photos and music and videos just because it is available then it will most likely be of less value to you (your second point). When you have one of something then it is valuable but when you have 20,000-30,000 of something then it loses its luster. Of course, since the time I wrote this blog post and now, 1Tb doesn’t buy you what it used to :-).

    1. Kelly Brown Post author

      Good question! The simple but frustrating answer is: it depends. In my blog example, approximately 86 pages of Word document constitutes one megabyte. That would be a short book but there is extra formatting in Word that takes up even more bytes beyond just the plain characters. In plain ASCII, a character is equivalent to a byte. If you have 50 lines of 50 characters each per page then each page would represent 2500 bytes which means that a 400 page novel would equal 1 megabyte. In a published book there will be some formatting involved because of fonts and other characters so you would be lucky to get a 200 to 300 page book in a megabyte. So, the answer to your question is one.

  2. ☺Nick☺

    I run 5TB and it’s full. Unless you watch 480p, 1080p and the upcoming 4k videos will eat a TB away.

    Time will pass, and you’ll find this thread a joke. I’m hoping to double my space in the near future.

  3. PaulG23

    You wrote that a terabyte = approximately 85,899,345 pages of Word docs.

    Okay, no human can read that much, but what if you were responsible for reviewing and determining what is important in 1 terabyte of data storage, where some is video, some is documents, some is audio recordings? And by important, I mean that someone’s life or liberty was at stake.

    Such is the dilemma facing criminal defense attorneys these days in some federal cases. The government dumps that much discovery, and more, on the defendant and his counsel. Then the prosecutors produce at trial just that carefully selected evidence which they think will help them convince a jury to convict the defendant. Judges typically don’t require the government to specifically disclose their exhibits, or expected trial testimony, or even prior statements from government trial witnesses, prior to trial. So, the defendant and defense counsel cannot anticipate and prepare for what is actually produced.

    The defendant and his/her counsel who have the temerity to go to trial cannot have reviewed all of the evidence, and are, therefore, taken advantage of, and surprised, at trial.

    Result: most defendants (95 – 97%) of all federal criminal defendants plead guilty before trial. Those who do go to trial lose 90 – 95% of the time. Those who lose at trial and then appeal, lose in more than 90% of the appeals. Overall result – in federal criminal cases, the defendant has approximately 0% chance of getting acquitted.

    Data overload is just one of the prosecution techniques deliberately used by prosecutors to destroy any chance of a fair trial.

    There oughta be a law – for example: cases where the prosecutor produces more than the equivalent of 100,000 pages of discovery materials, should be dismissed for violating the due process right of the defendant to get a fair trial.

    That would give federal defendants a little more of a fighting chance.

    1. David Bernazani

      Paul: Wow, I never knew about that. It doesn’t apply to my life or career (thank goodness!) but I will certainly remember it!
      Thanks for sharing.

    2. David Bernazani

      Paul: after reflecting on your post again, I just realized that once computers become intelligent enough, they could sift through those mountains of data quickly and extract the needed pertinent information. It’s just a matter of them being programmed to be discerning enough to know what to look for. True “artificial intelligence”.
      So perhaps soon, humans will be much more dependent on computers than we already are–or at least, those humans in trouble with the law!

    3. Lee Rutter

      Interesting take on the volumes of data that the attorneys and prosecutors (not to mention defendants). The more serious the crime, the more volumes of data including references and copies of other cases! I have witnessed something very interesting prior to computers being involved and that is watching one person sue another claiming they should receive more money for child support. The step-mother had convinced the father from the very beginning of the the divorce to keep a physical calendar and mark the days the boys were in his care. Each year, the calendar gets taken down from the wall and placed in a safe place. Sure enough, the mother sues the father and the step-mom reminds father of the calendars. Father takes calendars to court. Calendars are handed over to the judge and within 45 seconds, the judge denies the mother any extra child support due to the fact that the father was organized with a stupid calendar and the mother didn’t have anything to justify her needs. In today’s time, with volumes of data that the judge nor the jury will ever have the time to read, the one who is most organized and has the best data no matter the content, typically wins! There are some judges that believes no one produces reams of data that is junk which is why many of the world corporations get away with polluting or commandeering natural resources away from the public trust. The consumer or victim oftentimes do not have the resources or the ability to provide 500,000 pages of research study where 499,400 pages are filled with fluff. You are so right of the prosecutors tactics.

  4. isaac holiday

    I have 600 CD’s and I am looking to store them on a flash drive. What size flash drive I might to need to store perhaps @ 100,000 songs only?
    Thanks
    IH

    1. Kelly Brown Post author

      A rough estimate of 100,000 songs at ~5 megabytes per song would equally 500 gigabytes or half a terabyte. Another way to consider this is that a CD holds approximately 700 megabytes so with 600 full CD’s you would have 420 gigabytes or almost half a terabyte. In either calculation, you would need storage equaling about 500 gigabytes.

      1. Bill

        I just did it quickly in my head, but I think the OP was off by nearly an order of magnitude. If IH really has 600 CD’s (@ say 20 songs per CD) he or she has around 12,000 songs not 100,000! I just was doing this out of curiosity (and for possible future reference). But 600 CD’s would easily fit on, say, a 64GB thumb drive, but as you correctly point out it would take half a TB for 100,000 songs. So your answer was correct, but the question was not well-formulated (unless the OP just mentioned the 600 CD’s as a red herring. (Or did I miss something?)

        OK I did miss something. I was assuming 20 songs per CD, based on my experience with audio CD’s, like one used to buy at a record store. I am guessing you are basing your numbers on mp3 or something like that (which I never use) stored on HD CD’s . But according to your assumptions a CD holds 140 songs (or about 10 record albums)! If mp3’s are really that small, no wonder the sound quality is so bad I cannot stand to listen to them. 🙂 (Most of my listening is from 24/96 files).

        So, do I have it correct? And if so, when does anyone ever find the time to listen to 100,000 songs? Even if they were 3 minute pop songs that would take almost a year listening 24 hours a day, just to listen to all of them once. When would someone find time to eat, much less time to make silly responses to IT blogs? 🙂

  5. David Bernazani

    I have about 800 DVD & Blu-ray movies now, and by the time I retire and move to Romania in a few years I expect to have roughly 1,000. This is a lot of boxes. I am researching the possibility of transferring them all to flash drives, or some other kind of memory device.
    I would need at least 4 Tb of memory to hold them, so just one Tb doesn’t seem like a lot to me.
    But I’m worried about long-term storage, data degradation, and accessing them in a convenient way. So clearly I have more research to do!
    Thanks for your info.
    -Dave Bernazani
    Lafayette CA

    1. Kelly Brown Post author

      Dave, thanks for your comments. I have been in the business long enough that I remember working with bytes and kilobytes so my perspective has partly to do with my history. It could be that the terabyte is the new megabyte and I just need to catch up :-). Good luck getting everything moved over to storage. I see that they make a 1Tb flash drive now so 1000 DVD’s down to 4-5 flash drive is quite a space savings.

      1. Bill

        I’m on a roll today! I am “old school” like you. I remember deciding to lash out and get a second 5 1/4 floppy for my first PC so that I had a place to store my “large” files (MS WORD for DOS could fit on the first floppy–and it ran fine (I forget…weren’t they 256K?…lol) Haven’t we come a long way?!

        I am not sure what a “flash drive” is, but I have all my music on a 3TB external USB portable HD (no external power cord required). I am sure they come in 4TB, but maybe even 5 or 6 (or wait a week!) 😀 It fits in my pocket! That seems all the OP needs. And an HD the size of a men’s wallet would be a heckuva lot easier to carry overseas than 1000 DVDs!

      2. David Bernazani

        Kelly, just as a follow-up, last year we got priced out of our lovely apartment and decided to move early to Romania, so last summer we packed up all those DVD’s (along with lots of other stuff!) and shipped it all there. Looks like I don’t need to worry about storing my collection, but who knows? Some day I may want to, just as a backup.
        Meanwhile, a terabyte seems to be growing smaller all the time…
        I’m also sure that the next format for movies will be like that of CDs: not a physical item at all, but digital versions stored on an “iVid”— some new version of the iPod. It still may sound strange to some, but it’s only a simple matter of storage.

        1. Kelly Brown Post author

          David, thanks for the follow-up. I think you are right about the future of movies. Who says that you won’t have a 1Tb or 2Tb SSD on an “iVid”. They could even be removable so you keep a small collection of drives on your shelf. The other option is cloud storage but you need to have access to the cloud in order to utilize it. If I am backpacking in the wilderness area, I most likely will not have a signal to be able to access my movies. If I get lost however, accessing my cloud storage to watch movies will be the least of my worries :-).

          1. Matt

            For the most part, how much media does one really need immediate access, everywhere they are? And things like popular music & movies everyone doesn’t need a copy, if a streaming service has them, always be accessible, and gives an offline option. (Going camping and want a couple movies for the trip, just plan ahead, have them download the day before and watch them when you get there, or passengers watch them on the way).

            Even small removable drives (1TB compact flash cards – not too small physical size, quite durable) with a good cataloging system. Or maybe even a library system on your home network, where you buy a toaster size device, where it would have slots for maybe 250 SD cards in it. When you run out of space, just drop a few more cards into it.

  6. David Bernazani

    Kelly,
    Yeah, I remember floppy discs, and even life before VHS tapes. My first computer didn’t even have memory; every program (read: game) I wanted to run, I had to insert a floppy to run it.
    Heck, I remember my first computer class in high school, we saved programs on a paper punch-tape ! The computer wasn’t even actually at school; we were hooked up to a university via some kind of early pre-internet connection, and used teletype machines to communicate with it!
    Memory was at a premium. I’m astounded today at how much it’s increased. A terabyte in a flash drive is like something out of Star Trek! And my iPhone probably has more power than the entire Houston control room for the Apollo moon missions.
    Cool times, huh?

    1. Bill DeSantis

      I don’t know the age of this thread but I just bought a two terabyte thumb drive on Amazon for $30. I’m 65 years old I’ll never fill it, but I have numerous things to back up. I have a few computers to back up and so on and it’s very easy to just carry this little thumb drive with me that has a key ring attached to it and have my PC laptop and my wife’s laptop all at a moment’s notice. Just plug it in, cuz someone’s computer which almost everyone has somehow somewhere. And then you have it all right there with you easy peasy. This is March 2nd 2022 just so you know. Bill D

  7. Kal

    I think this is an interesting article. It looks like I may be among the youngest on this thread. You said “maybe the terabyte is the new megabyte” and that you haven’t caught up or something to that effect. I think that’s the nail on the head. You are thinking of data storage in an antiquated way. You’re thinking of cds and floppy discs and how much a typical song, movie, or document takes up without recognizing as that a movie, audio file, or document is not a fixed measure. Someone else mentioned 4K video, well with advances like blue ray, 4K, mp4, upcoming wearable technology, etc we will need more data storage for the same amount of content. Example: 100 songs in mp3 format will be different then the amount of storage for 100 songs in mp4 format.

    Hope that all makes sense I just stumbled across this thread and thought I’d put in my thoughts.
    -Kal

    1. Kelly Brown Post author

      Hello Kal,

      Thanks for your thoughts. You are correct and I probably need to update this blog post. I was talking with someone the other day about Blu-ray, 4K and mp4. They all require a lot more storage space. I see 1 Tb flash thumb drives now as well as 2, 3, and 4 Tb backup drives from reliable suppliers such as Seagate. The price is coming down as well. I wonder however whether storage makers are are working to keep up with the new content formats or are content providers developing new formats to take advantage of advances in storage technology. The real question however is when will we see the first consumer 1 petabyte drive? Do you care to make any predictions? :-). Thanks for your note.

      Kelly

      1. Bill

        If a petabyte = 1000TB, I would bet 2020.

        Do you remember MS-DOS decided on 256KB for “base memory” because it was such a high number that consumer PC’s would never need more than that? (of course you do…that was a rhetorical question :D) I think that assumption was outdated about a week after it was implemented…

        I do not think it has been all progress though: MS WORD for DOS did just as good a job as my current WP for my purposes, and it did it faster and more efficiently, and it fit on a floppy. However, I run a music server full of hi-rez music on an XP machine, and it sounds as good as vinyl (for files that were digitised from vinyl) and the PC was free ( a friend was throwing it out) and the DAC runs rings around the best redbook CD players and it costed about as much as a middle of the road consumer CD player. I already mentioned my HD… I was just reading about DACs that perform nearly as good as mine, but are the size of a cigarette lighter!

        So, there have been advances. But also setbacks. Not a day goes by that I do not curse my Windows machine (usually 30 seconds be fore it crashes), I devote about a thousand times more memory than my first computer had all up just to control viruses, and whilst formats like mp3 allow people to carry around thousands of “songs” it makes ones ears hurt listening to them…

        That said, with a retired xp machine, A$100 portable HD, and a A$1000 DAC one can have a digital front end for a HiFi setup that rivals a $10,000 turntable!

        1. Kelly Brown Post author

          Bill,

          Thanks for the great comments. Yes, a petabyte is 1000 terabytes or 1 quadrillion bytes. So, the question remains, how did we get from 256,000 bytes to a trillion bytes and a future quadrillion bytes. I think a lot of it has to do with the physics and the fact that we can continue to pack more information into the same space and we are moving from mechanical disk drives to solid state drives. I know we will hit physical limitations in the future and will have to think smarter about how large of programs we make. We may have become sloppy about how much memory and storage we use for common programs as your illustration of Word points out. Granted we do have nice graphical interfaces now that we did not have in MS-DOS versions but I fear that we have let these programs balloon “because we can”. The physicists and electrical engineers designing and creating integrated circuits have done us a favor by continuing to push the physical limits but they have also made us lazy.

          I am envious of your music server. It sounds like a great setup that serves you very well.

          1. Bill

            All too true. It is amazing how much waste there is “because we can”. Although I admit I like GUI’s for many applications. I keep threatening to go LINUX…

            And yes, I love to make a silk purse from someone else’s sow’s ear. After a good friend gifted the DAC, the rest was easy: a discarded XP machine and a $200 5TB Seagate drive (plenty of extra space). The throwaway society is good for us folk who intercept perfectly good junk before it ends up in the tip! Also I should plug JRiver: a great little music server program for US$50; I could not have achieved the quality I have w/o them… 🙂

          2. Bill

            Hi Kelly

            BTW: just curious: has anyone calculated the theoretical limit for info, e.g. assuming that one bit of info would need at least one electron to store it, how much info is it possible to get on say a cm2 chip? Or maybe the limit is the size of a silicon atom? Have you heard of any such calculations? Thanks 🙂

          3. Kelly Brown Post author

            Hello Bill,

            Great question and the answer is “I don’t know (yet)”. I will do some research this week. I used to work in integrated circuit design a number of years ago and there are a lot of variables involved in creating denser and more complicated chips, some dealing with physics and some chemistry. My brother works in the semiconductor design/processing area so I will ask and let you know what I find out.

            UPDATE: Bill, here is the opinion I got from experts in the field on how much more storage we can fit on a chip:

            “As far as storage, there is still quite a lot of room for increased capacity. 3D NAND and 3D Crosspoint are new memory products that go vertical instead of horizontal to build the cells so you can put a lot more bits per square inch of silicon. Both have been proven to work and both have started to be manufactured in their earliest forms but there is much more room for further improvement with known paths. That being said, you still need to be able to address each bit, either directly or indirectly so that appears to be the constraint to me. I think petabyte thumb drives are possible but we’re still some years out from that.”

            So, think three dimensional in terms of silicon chips. Of course, that comes with a lot more complexity in design and processing but it does allow us to keep pushing the boundaries of how much we can store on one device. Now you need to start thinking about how many songs you will be able to fit on a petabyte drive in the future 🙂

            Kelly

          4. Bill

            Thank you, Kelly. I like forward to your response. Mainly curiosity, but I am a philosopher of technology by trade, and such “facts” may come in handy for my engineering student lectures… There was a book published in the 80’s by a philosopher called Christopher Cherniak call Minimal Rationality that estimated how much information could be stored in the entire universe. At the time all the major theories of rational thought (going back to Plato) assumed that an infinity of information was available. It was a revelation when Cherniak revealed how small the universe actually is when compared to infinity!

            Mine is a similar sort of question, but I think much more practical… 🙂

          5. Mike

            Indeed, we are approaching physical limitations with the amount of information we could store on a solid block of metal. I think that quantum mechanics holds that promise, especially with recent advances in quantum computing; storage may just begin to increase exponentially in the near future.

  8. David Bernazani

    Kelly, do you mean flash drive or hard drive? I believe the highest-memory hard drive (as of 2015) is/was 16 terabytes. It appears that Moore’s law is reaching its theoretical limits (computer memory doubling every 18 months), but it should hold for at least a few more years, which means that 16 Tb hard drive should reach a petabyte in roughly 9 years. But I wouldn’t be surprised if we have a 1petabyte hard drive much sooner. As for a flash drive, I imagine it may take a little longer, as flash drives always seem to lag behind hard drives in memory space.
    And by then I think we will be approaching the limits of memory storage– what do you think? I think a petabyte was what Arthur C. Clarke figured would hold the entire lifetime’s worth of memories, emotions and all other data from a human brain (as written about in 3001: The Final Odyssey). Or maybe it was 5 Pb? Must check on this…..

    1. Bill

      Like you I think we will see acceleration in storage size increases. (See my prediction above) So many people want to be able to carry every video ever made around in their smart phones! The only thing that will slow it down will be if people decide that cloud storage is a good idea. (I do not…) I guess there will be another limitation caused by the size of an electron, but although I really have no idea, I imagine we are not close to that yet! I think by the time that we get 1 PB consumer storage, though, it will be flash. The technology is diminishing in cost exponentially (I remember lashing out and buying, I think a 640MB, thumb drive for $130 in Singapore, where were cheap. This was a lot of money (I could have bought a cheap laptop in Singapore for 3X that), but it was a HUGE amount of storage (I still use it as an archive for old WP files) but it was a bargain at the time. Now I hesitate to pay A$30 for a 32GB thumb drive, because I think next week I might get a 64GB for that!) And really I hope in a year or two moving parts for storage will be ancient history! I would think, perhaps naively, well-built solid state would have to be more reliable than motors and bearings?

      1. David Bernazani

        Bill,
        Yes, I also want to see the end of moving parts for storage devices. This is why I’m waiting to get a new laptop/tablet until the 500gb SSD prices come down a bit more.
        But your idea that people want “every video ever made” in their phones made me think:
        It’s not as crazy as it first sounds.
        First of all, no one wants EVERY movie or video; everyone has certain tastes and “filters”, choosing just the ones the really like. (And think of how many BAD movies there are out there!) That’s why I have 800 DVDs at home and not thousands, which I could easily have these days, now that you can find literally thousands of them for sale online for as little as 75 cents. I’m selective (picky) about which ones I want to own.

        And I already DO have about a thousand songs on my 64g iphone– again, only ones I like. Today there are phones that could hold, what, ten thousand? Twenty? More? If not that many yet, there soon will be. And 20,000 songs is probably the max almost anyone could possibly want on a phone.
        Now apply the analogy to video. It’s all just a question of storage, and if phone storage keeps doubling like it has been, in a few more years, your “crazy” idea of putting a thousand movies on your phone sounds downright inevitable!
        And as I don’t know many people who would have the time to watch more than a thousand movies on their phones, I don’t think many would want much more than that. Which means your “crazy” dream of “every video ever made” on a phone would, in a sense, become reality. And not too far in the future at all, my friend.
        See you there!

        1. Bill

          Hi David

          Yea, I probably already have more music than I can ever listen to in my music server–3TB, probably 300K songs (mostly Hi-rez–I prefer quality over quantity!)

          It is funny though, I have music albums (vinyl–lol) that I may have listened to hundreds of times but I could probably list in a few minutes the number of movies I have watched twice, and I think there are maybe a half dozen I have watched more than twice. Even though storage is cheap, it would seem a waste for me to have too many movies. Most of the movies I have watched twice are because I have wanted to share with my daughter (who like me has eclectic tastes) cult classics I love. It is pretty easy for me to find movies like those in the library…

          My question–and it is just curiosity–do you really watch hundreds of movies over and over again, or do you have a professional interest that makes it useful to have all those videos? I have a friend who wrote a book on philosophy and film: he owns more movies than I have ever watched! 😀

          Anyway…feel free not to answer, if the question is too personal! 😉

          I wonder if it will get to the point where every phone will come loaded with a few out of copyright songs and movies–say 1000 movies and 100,000 songs–just to entice new customers to download more!?

          1. David Bernazani

            Bill,
            Sure, I don’t mind answering. I’ve always loved movies and watch several each week. We cancelled television and only have my movies (and a few good TV series on DVD– Frasier, House, Sopranos, Game of Thrones, etc).. I’ve probably seen almost all 800 of my movies at least twice, some many times. When I’m in he mood to watch a movie again that I know, I like to own it. I don’t have
            a good internet connection to stream videos; instead, I rent a few new movies from Netflix each week (by mail) just to keep up with them all!

            And we are retiring to Romania in a couple of years, where there will REALLY be no television (worth watching, anyway), so my collection will be all I have for the rest of my life, except for new movies I’ll watch online on Netflix or something.
            So that’s why I’m collecting good movies now. I’ve already bought almost all the current movies I want, and will mostly just get any future ones that I like.
            So that’s my deal with dvds!
            -Dave

  9. Bill

    That makes sense, David.

    Funny I occasionally go on a movie spree, but i just nip down to the local public (or local) library! (it is good to get out of the house!) I love movies. much prefer the big screen (cinema), but it is hard to see even 1% of what I want to there… We have great TV here in Oz between the two national broadcasters (now with at least two digital stations each) there is plenty of quality content. They introduced a limited amount of commercials a few years ago, but not enough to be obtrusive! I have resisted the temptation to stream over the internet direct to the TV, but I often watch documentaries on the computer. It would be too easy to stream direct to TV, because I am pretty sure my music server PC has wifi, but I am afraid if I turn it on, my life beyond the couch would end!

    I was born with commercial TV (1961) and until I was 15 I watched daily (never missed an episode of the Brady Bunch) I decided I was wasting my life and stopped watching aside from my religious weekly viewing of Saturday Night Live! At university i had no TV. In fact I have never bought a TV in my adult life. I currently have one that was given to me. Mainly it is a monitor for my music server, but I do watch the occasional football match (rugby league) and about 3 movies a week! It is funny. I have sometimes gone a decade or more without even knowing what was on TV. I have a fair familiarity with South Park and the Simpsons. I totally missed Seinfeld (although I have seen about six episodes total and I enjoyed it.) Sometimes I borrow TV videos but they never seem to have SEASON ONE! lol Ther was a really good show produced locally called Rake. understand they tried to produce a US version that flopped. f you want insight into Aussie humour, get a copy of that. I think there were only 3 seasons… Available from the ABC (Australian Broadcasting Corporation–the national broadcaster; like PBS) Shop online 🙂

    For a while I was religiously watching House (the season that ended with him performing surgery on himself under local in the bathtub.) I really enjoyed watching the writers having fun making the scripts more outrageous each week. When I retire I will probably obtain a set of House and I have also heard the Sopranos is good… Happy viewing!

    1. Bill

      I re-read may comment and realised I left out the word “university” when I stated that I got my videos from the “local library” twice 🙂

  10. David Bernazani

    Yeah, I grew up in front of the boob tube also, (except during the ages of 5 – 9 when we lived in Greece which had virtually no television shows at the time), but as an adult my tv watching has been spotty at best. I loved the original Saturday Night Live cast but totally missed the years with Eddie Murphy. I didn’t know why everyone was saying “Otay”.
    And although I’ve never seen every episode of a show on live broadcast, when you have them all on DVD it’s great: you can watch them whenever you want, as many as you want. You can have a “Scrubs” marathon all weekend if you like! My wife & I spent one winter watching every episode of “Friends” and “Downton Abbey”!
    I also have all or most of “The Simpsons”, “Big Bang Theory”, “M*A*S*H””, “Breaking Bad”, Seinfeld and Nurse Jackie.
    I’m starting on some of the Star Trek series now (Voyager and DS9). I’ll have a lot of classic American television to watch for years to come!
    Oh, and I found that I can rent a show called “Rake” on Netflix, but it stars Greg Kinnear, who I believe is American, but is see that Miranda Otto is also in it, and she’s Australian, I think. It’s from 2014; is that the show you mean?

  11. Julie

    I have 2600 4Kultra photos and videos from a month in scotland – mostly pictures – a bunch of short videos. What is the best way to preserve these? Just keep them on the card they are on, and buy another for my camera, instead of trying to put them somewhere else, and delete them from the card in the camera. I’m trying to transfer them to a ‘passport now, but am afraid I’ll lose the 4K quality, and that there won’t be room for them on there?

    1. Kelly Brown Post author

      It sounds like a successful month in Scotland. I am going to have to do my homework to figure out how much a 4K picture or video eats up in terms of storage. Here is a good article from DPReview: https://www.dpreview.com/what-is-4k/. The Passport from Western Digital now comes in 1TB – 4TB sizes and use USB 3.0 which allows for a faster transfer rate. I am not sure what size Passport you have but if you were able to fit the photos and videos on the SD card in your camera then you should have no trouble fitting them on a Passport. The Passport is a great long term storage mechanism. The only thing better would be a solid state drive (SSD) which has no moving parts and faster transfer. I saw last week that Kingston will introduce 1 and 2TB thumb drives (SSD) next month which will most likely be crazy expensive. I will let you know what I turn up on the storage requirements for 4K unless one of my readers jumps in first. I have been studying up on compression algorithms lately which would make a difference on whether you retain the 4K integrity when transferring the photo from one medium to another.

  12. Matt

    I used to keep increasing storage until I hit 700-800GB about 4 years ago and been too busy to take too many pictures, or watch videos or listen to music and it really slowed down from there. I bought a 1TB fireproof hard drive to back things up on and haven’t got it maxed out yet. I think its come to the point where most people don’t even store that much data, especially if they are over 35. Somethings like iphoto seems to just waste space with extra sizes and backups, even if you don’t even want that certain picture anymore.

    As for hard drive sizes, I don’t think 1PB will come in 2020, maybe 200-300TB. SSD drives don’t really have a limit…just make a 3.5″ drive, cram it full of chips and it could possibly be 1PB now.

    I think there is a curve of peoples space needs. Back in the late 90’s to maybe 2007, I was always hitting the limit and needing more space. Now there is so much space that its no longer a concern to me. (I do hate having too much stuff because it makes backing up more time consuming and complicated). Unless you are heavily into video or on a portable device, space really isn’t a concern anymore to most poeple.

    1. Kelly Brown Post author

      Thanks for your thoughts and contributions to this blog topic. I think that you are correct that space is no longer becoming an issue. That means that a large hard drive in a computer is no longer a selling feature. Most of them come with at least a 1TB drive which for most people and applications should be adequate so it does no good to tout a computer with a 2, 3 or 4 Tb drive. Besides that, there is also cloud storage as well so not everything is stored on a hard drive anyway. The world is changing indeed.

  13. David Bernazani

    Kelly,
    Yep, I think the “curve” of demand for increasing storage space for personal use is flattening out here in the post-terabyte age. I guess most simply don’t need more than a few TB’s. But that kind of storage is only available on hard drives so far, as SSD (flash drives) have lagged behind. And as was discussed earlier, some of us would prefer not to use a medium with moving parts that can have problems (read: crash), so I’m still waiting for that future laptop to appear with at least one TB of SSD– at a reasonably cheap cost. Yes, they’re probably available now, as someone is sure to point out (I haven’t checked lately), but I’m sure they’re still quite expensive.
    And, as I originally posted about, I would love to be able to download all 860 of my movies onto it, so I really need more memory– maybe 4 or 5 TB. So for some people, it still IS important for manufacturers to use that type of storage as a selling point. I want the biggest they have!
    So I guess I still have a bit of waiting to do. But I’m patient, and can wait a couple more years. That’s the great thing about memory storage– you can simply wait a while until it’s enough for your needs!
    PS Update: I’m spending the summer in Romania and I was correct; there is no television worth watching here. So when I retire in 2 years and permanently move here, I’ll have to bring every movie and tv show I may want to see again for the rest of my life. That being said, Romania has some of the fastest internet speeds around, having upgrades the country to 4G, with plans to have 5G by 2020. So streaming will also definitely be an option, and perhaps soon after I lug all those DVD’s here, they’ll be as unneeded as my old CD collection! But it’ll still be nice to have the old hard copies; who knows? Maybe they’ll be worth selling here!

    1. Kelly Brown Post author

      Hello Dave,

      It is good to hear from you again. I am glad you are enjoying your summer in Romania. I will keep my eye on storage manufacturers to see if the timing is right so that you can leave your DVD’s when you go there permanently. I am fascinated that Romania would have fast internet speed but I am guessing (speculation) that they did not have legacy networking to deal with. Rather than develop from dial-up, they perhaps started with a faster platform and it was easy to upgrade from there. There are some advantages to not having legacy systems and processes to tie you down but that is a topic for another blog posting :-).

      Kelly

      1. David Bernazani

        Kelly,
        I was just rereading this blog and thought I’d give you another update. It’s been a year since I was in Romania, and here back in the US, I’m still collecting movies to bring there when I retire (still at least a yearawsy).. Tonight I just broke 1,000 DVDs, when I scanned a couple of Rick Steves travel videos into my app.
        It’s called Sort-It, and it’s the best, most helpful app I’ve ever downloaded for free. With it on your phone, you can scan the barcodes of your movies, and it automatically uploads all the info about them ((including front & back case covers), and sorts them alphabetically or however you want.
        Anyway, now almost all my new DVDs are Blu-Ray (I haven’t made the leap to 4K yet), which will take up even more memory should I ever want to copy them all on some storage device.
        Doing something like that sounds like so much work, I have no real interest in trying it, but if there should come a product— and I suspect there will— that quirky and easily stores people’s DVDs and Blu-rays, and played them back, then I’ll pronably join in.

        1. Kelly Brown Post author

          Thanks David for checking in. It sounds like there is a potential market (I am sure you have like-minded friends) for an easy storage product. I am not familiar with Sort-It but I will have to check it out. It would be great to have a similar product for cataloguing pictures but unfortunately those do not come with bar codes or liner notes except your own memory.

  14. Christa

    I have a household of 8 and we all use wifi/Internet. I work from home 8 – 10 hrs a day on Internet I have 2 gamers who are online constantly and the others use wifi quite a bit. Is 1 terabyte enough for my family?

    1. Kelly Brown Post author

      Christa, it sounds like you have a busy household. You may have a lot of data passing over your wi-fi but 1 terabyte of storage should be sufficient for your family unless you start to build up a library of movies or videos or even music. As long as you are streaming content, you are not necessarily storing it so it would not count towards your 1Tb limit. You will most likely be limited in internet bandwidth long before you run out of storage.

  15. Tom Cunningham

    David, I just came across this article ant it is interesting how much and little has changed since this entry started.
    My observation is that most people are getting their content from online sources like YouTube or Netflix. In fact a few years ago my I.T. coworkers were going to online storage of their own content so they could get to it from multiple devices.
    I see this and the ‘cloud’ concept as the major paradigm for the next few years at least.
    My biggest issue seems to be hardware failure and backup time. I’ve lost a couple of hard drives in the last year, some of which were not as well backed up as I should have (shame on me).
    I personally do DJ work so I have several thousand songs on CD and stored as mp3 (320kb). I remember not only paper tape but hollerith cards and when 4k of ram was typical.
    I probably have about 10-12 TB of hard drives, but I know very few people who have more than a few TB.

    1. Kelly Brown Post author

      Hello Jacob,

      Thanks for the note. This topic is almost a nightly conversation in my house due to aging parents so I am thrilled that you have created this service for your clients. It is sorely needed. I will send you an email so we can discuss further.

      Kelly

  16. Lisa

    Hi Kelly,
    I have just found this blog and think I will read some of the other ones plus the newer ones.
    I was wondering (as with Dave), I have a stack of DVDs that I have been hoping to get transferred into a hard drive just in case I have short-term relocation (or even permanent/long-term) and I just won’t have the ability to move them….do you have a suggestion for a program that can be used to “transfer” from disc to digital?
    Thanks, Lisa.

  17. Lisa

    Hi Kelly,
    I just stumbled across your blog/website and found it quite informative. I plan on reading more of it.
    I was wondering if you could suggest for transferring DVDs onto portable hard drives as I am hoping for short-term/long-term relocation across Australia, and I won’t have the ability to relocation my extensive collection with me.

    Cheers, Lisa

  18. Michael

    I like what you have said, and I agree, if you are keeping a terabyte of logs on hard drives, I would sure hope they had a workforce of over one hundred. However, as a gamer (I hate that phrase) and someone who handles 4k video, I can blow through a terabyte hard drive only two weeks after I have reinstalled windows. Once we see 1TB SSD’s at 200 dollars, then I will shell out and buy a couple of those, maybe I should invest in gigabit and bulk cloud storage?

    1. David Bernazani

      Michael: Exactly what I’m waiting for! When 1Tb SSD drives on laptops are a little more reasonable, I’m getting one, and hopefully it’ll be the last computer I’ll ever need. (I’m 58, bin case you were wondering!)

  19. Matt

    As someone else pointed out about information overload, I think there is just a point where we’re saving too much data. I do computer repair/data recovery and have started finding computers with 60,000 to 80,000 photos taking up 300 to 400GB of space. They would go to a sports game and take 200-300 pictures or a play. Or have some fall pictures with the family and take 75-100 pics. Who has time to look at all those photos? How much of our lives need to be documented? Is anyone considering the time it takes to catalog/backup all that data? How does one even find a particular photo with that much there? Video is even worse. You may be carrying around Terabytes of family videos, but is anyone even watching them?

    1. David Bernazani

      Matt, I agree, it’s ridiculous how people thoughtlessly store EVERY SINGLE boring photo they’ve ever taken. I constantly winnow out all the bad photos almost as soon as I’ve taken them, saving only the best or most interesting. Think of them as if you were someone else, forced to look at them all. Would you really want to look at 20,000 photos? Especially when most of them are probably very similar to each other? (See uncle Joe. See him standing next to Grampaw. Now see him on the other side of Grampaw…..yawn!)
      I also go back through them all once in a while and delete even more; time helps you realize that you really didn’t need 6 photos of your cat that day when one or two will do. I have, I think, between 100 and 200 photos on my phone; the total summary of my life, taken in the last 2 years.

    2. Scott

      I carry this sentiment as well, Matt!

      I’ve read that about 300 hours of videos are uploaded to YouTube every minute. I can only imagine how many Instagram photos flood their servers every minute. And I have friends on Facebook that upload upwards of 40 pictures of a family dinner, and maybe 80 pictures from a trip to a museum. And professional photographers throwing out a hundred or so photos per day. When I think about RAID arrays filling up so fast, I think about how fast data centers need to be built to keep up and the feeling that comes over me can best be described as a sense of overwhelm. Are our storage technologies improving at the rate needed to keep up? When will we have so many indexing tables to catalog it all? When will we exhaust our URL’s? I imagine YouTube would re-institute the 10-minute limit from back in the day. Or rather, they should.

      Even if we can keep up, I feel that in the next 30 years there will be too much of everything. Too many movies, videos, shows, music, podcasts, cat pictures. I then get the feeling we’ll be living in a world of quantity, where the quality is spread too thin, adding to my aforementioned sense of overwhelm. I just feel like people really need to slow down. And to realize that no one’s gonna give a hoot about the drink they’re having showcased on Instagram 10 years from now. In a reply to a previous comment on this thread, Kelly mentioned that we’re “‘ballooning out’, because we can.” I feel that applies properly here. Everyone uploading photos because we can, and especially because we’re addicted to the dopamine response that comes from getting those likes and hearts on our pages.

      I do apologize if this comes out more as a rant than a philosophical waxing, but these are the thoughts that have been haunting me enough to Google and find a thread such as this. I’m glad to know we seem to be keeping up in terms of storage, but can we trust the majority of people outside us to eventually clean up their acts? If the tech giants decide they can’t keep up and declare upload limits, I believe people will react negatively.

      1. Kelly Brown Post author

        Hello Scott,

        Thank you for your thoughts. When I wrote this blog post I did not envision that we would have a philosophical and technical discussion on the merits of storage but it is indeed appropriate. The term sustainable comes to mind and I don’t think that our current trajectory is sustainable. We can continue to build new data centers but at what cost? I recognize that data centers need to expand based on our migration to cloud based computing and storage but if the data centers are filled with pictures of last night’s dinner and cat videos that cannot be indexed then we have a mess. We need to start talking about responsibility OR we need to move to application such as Snapchat that house images only temporarily. I am glad that we are starting the conversation here. Thanks for being part of it.

      2. David Bernazani

        I am living what you guys are talking about, because here in Ashburn, Virginia, a new suburban community, the housing tracts are interspersed, strangely enough, with massive data centers. These huge, windowless, imposing edifices seem to sprout up almost overnight, around every other corner. You wouldn’t know what they are if you were just visiting because they put no signs outside them. I only know because I live here now.
        Each one must cover about five acres of ground, and they’’re all several stories high. The amount of data in them all combined must be nearly inconceivable, yet they continue to build more.
        I only wonder how much access the government agencies have to them. It must be a coincidence that Ashburn is just outside Washington D.C., right?

  20. Teri

    umm… 1TB holds about 250 movies? Movies that are bought are bought to save and re-watch. To re-watch your favorites, just re-watching one movie a day is 365 movies in one year.

    And I like new movies also. I watch around 2 movies a day. That’s not much. 1 1/2 hours each, that’s only 3 hours. How much time does someone spend watching TV per day? Serious movie buffs are more inclined to skip the cable TV and stick with movies. Serious movie buffs can easily own far over 2000 movies — and to keep track? Its just a simple list of titles in excel. 250 movies stored on a computer with 1TB…. that’s a long jump away from being enough.

  21. BlueNexus

    So is my 26.5 TB space too much? Probably so. But considering most of the drives came out of old computers that were either junked or upgraded then I have to say no, it’s not. Will I ever fill it all up? No.. But there’s a reason for that too. By the time I have enough stuff to fill up such drives the average drive capacity will be so much higher, that it will be like holding on to that massive 3.5 GB (no, not 3.5TB) drive that I had back in the early 1990’s. That drive cost $350 which was a lot more money back then. $100 per gig(USD.) I eventually intend to build a Linux based NAS system just for the fun of it. But I doubt I will do a RAID system and will probably just name them all individually or perhaps create a backup system with half of them. All in all, I have 17 HDDs most of which are SATA and 3 SSD’s with closer to 28 TB if I do not consider the extra 48,576 bytes per megabyte that many companies simple round off.

  22. Pingback: Let's Go Shopping: DropBox or Google One? | Marina Reznor, Author

Leave a Reply to David Bernazani Cancel reply

Your email address will not be published. Required fields are marked *