A call For The Home Media Network 1 Gordon Bell and Jim Gemmell

4 May 2001

Technical Report

MSR-TR-2001-52

Microsoft Corporation

One Microsoft Way
Redmond, WA 98052

“The best way to predict the future is to invent it.” – Alan Kay

1Introduction

Typical homes have a plethora of answering machines, boom boxes, cameras, computers stereos, telephones, and TV sets. This congeries is interconnected via at least four, independent networks carrying audio, data, telephony, and TV. Some homes add intercoms, home theatres, surveillance cameras, and home automation controllers. We talk about home audio-visual (A/V) systems, but applying the word “system” is generous to the extreme. For the typical consumer, it is difficult (or impossible) to interconnect all the possible devices. In the living room alone, the proliferation of remote controls demonstrates the lack of integration (Figure 1). True, the TV is usually connected to a cable, satellite and a VCR/DVD. The stereo also has a number of connected components. “High end” homes may even centralize audio sources and amplifiers and run speaker wires to every room, including providing a receiver and player for each family member. However, to truly integrate all home media requires a scary collection of special equipment, a sense of humor, trained operators, and a full-time maintenance/user consultant (the responsible person in the family).

Figure 1: Remote controls from one living room demonstrate the lack of integration in home A/V “systems”.

The computer is starting to show how all media can be integrated. In essence, A/V content has been consumed by or “converged” into the computer. Consumer grade PCs can play CD-quality surround audio, play/rip/write CDs, and play/write DVDs. They can tune and record TV. They can also store and print digital photos, display art, and record phone messages. However, current PCs are not the answer, because they are not usually welcome in all rooms. They tend to be big, ugly, and noisy (just painting them black isn’t the answer). They take too long to turn on. Also, the benefit of a PC’s flexibility and extensibility comes at the price of management and maintenance, making few people willing to maintain many of them. Their complexity is comparable to the multiple remote controls and the interconnected A/V equipment that occupies most listening/viewing stations.

We believe that a single, home network that connects a PC-based server (or servers) to specialized media appliances (and other devices) is likely to evolve. Smart speakers should connect directly to the network and just play the appropriate sounds served to them. Smart networked monitors should let us watch TV, surf the Web, and display our art, ambient environments, photos, and various video content. Microphones and cameras should connect to the network for communication, to allow telephony and videoconferencing.

Instead of needing to find the right remote control for the right device, any remote control should command the unified system (e.g. pressing pause on any remote control will pause the media in the room). However, current remote controls do not have a rich enough user interface (UI) for all the potential of the home media network. A wireless keyboard, or a 3D pointing devices such as the Gyropointer mouse allow richer UIs. Even better, a wireless tablet PC can support a very rich UI, and can also display and capture media (e.g. record audio). We believe the tablet PC will be the next generation universal remote. Farther in the future, you will be able to control the system by speaking to it, or gesturing with your hands.

To the technophile, a digital home media network is obvious and inevitable. There are technical and political problems (e.g. copyright protection) to be solved, but they appear solvable. Digital media has already invaded the home in CDs, DVDs, PVRs and portable digital music players – surely integration can’t be that far away, can it? On the other hand, the average non-technical person asks, “why bother”? Why bother, indeed, when existing wiring doesn’t do the trick, much of the content is still analog, and the desired gear is priced for the high-end buyer?

The remainder of this article provides motivations of why we should bother, how and what future content will be distributed and stored, what the hardware/wiring requirements are, and interim steps in the journey towards the all-digital future of the Home Media Network. One critical interim step is the development of “digital transformers” that connect existing analog equipment to the home media network. Of particular interest is the Digital Home Entertainment Center (DHEC) that would almost completely replace all components of the current home entertainment center.

2Why bother?

To begin with, we enable content to go where it has never gone before. That big-screen display that is turned off right now is just a big, ugly, gray blob. What if it was a work of art? A slide show of your favorite photos? A fireplace? A passive or interactive aquarium? A window looking out on a tropical beach or at the scenery from the dining car of the Orient Express? Imagine lying on your bed and hearing the sound of the wind on a beach, while peering at the stars on the bedroom wall or ceiling displays.

Perhaps you watch your favorite TV show, and it mentions a Web site. You click “bookmark” on your remote-control. After the show ends, a PC or the TV can bring up the TV bookmarks for web surfing. You watch a baseball game and “bookmark” some plays. Then email the bookmarks to a friend, who had the game recorded while working late. Coming home that night, she is able to watch the highlights you selected. The next morning in bed, you bring up the “Times” on the ceiling display. Then you watch a video-on-demand lecture while exercising. A camera and microphone co-located with a large monitor allow you to videoconference.

Integrating media with the world of computers doesn’t just mean Web surfing. It means software that takes the “management” out of media-management. Lets take a moment imagine a world without arcane, nested menus; that doesn’t force us to memorize channel numbers, song numbers, radio frequencies, or the special key-combination required to set the time (or record the message, or…). The software needs to learn the kind of music or programs we like. One can ask for songs selected by a favorite DJ, but when the DJ picks a song the system knows you don’t like, it will skip it or play something you do like from your own library instead. Once a song is given the thumbs down, there is no reason to ever hear it again (at least not in your home). It will also learn what kind of photos and art you like. It will dig through the 99% of your photos that you will never bother to put in a photo album and bring them up to let you identify keepers.²

Imagining the possibilities is exciting. Some users have even wilder imaginations that us: a common feature request for UltimateTV is to be able to fast forward a real time broadcast channel! Rather than commenting on viewer intelligence, we’d like to point out the unlimited expectations in those requests.

But enough of being imaginative. Instead of us defending the motivation to switch to digital, lets put the champions of the status quo on the defensive. Why keep analog? It is not inherently cheaper. It is lower in quality. Let’s face it, analog has reached the end of its useful life, and it is time to bury it. Lets not use HDTV, which will only prolong the life of analog (The MPEG format that HDTV uses is fine – we are referring to HDTV as a transmission scheme, and as a display standard). This means a complete change in every aspect of audio and TV distribution, display surfaces, controllers, networks, computers, and how all of these work together and are controlled. To achieve this goal requires effort, determination, new standards – and getting started!

3Building the Home Media Network

The home dictates another important difference – the “mainframe” architecture makes sense with “thin clients” or “network computers” and centralized configuration and maintenance services. This idea is unpopular with PC users who want the option to buy different computers for different applications to keep control. In WAN environments, bandwidth limitations are a barrier. Within the home, these difficulties disappear. Most of us want to maintain just one or two machines. Bandwidth in the home should be plentiful as we show below. Hence, the home is just fine for thin clients (including TV sets – we are already familiar with “head ends” for cable or satellite).

These thin clients are “smart” in that they are configurable, connect to the network, and can be updated. However, they are not meant to be an extensible, general purpose platform, like the PC, with many unbounded functions. Such flexibility leads to unexpected combinations that makes performance unpredictable and maintenance difficult.

In an ideal world, all media devices are completely digital and all transmission is via an Internet Protocol (IP) network. However, we foresee a very long interim that involves both legacy analog and digital for transmission and viewing. To cope with this mix, we envision each analog receiver or television set has a “digital transformer” that converts digital to the analog world of the amplifier/speakers and TV sets or monitors. For example, Voyetra’s Audiotron also marketed as Gateway’s Connected Music Player network device drives an amplifier with speakers and plays audio content from any server on an Ethernet/IP network. The next device we would expect in the genre is a Videotron, or Connected TV Player that plays TV content coming over an IP network. Similar digital media appliances will be forthcoming for cameras and microphones. Of course, there will also be combination units, like a speaker/mic combination as an IP phone. Figure 2 shows a home with media servers, digital transformers, and digital media appliances.

Figure 2: The Home Media Network with digital transformers to support legacy analog devices.

We propose one digital transformer that will be critical: the Digital Home Entertainment Center (DHEC). The DHEC will almost completely replace every A/V component by integration into a single unit (see Figure 3 and Figure 4). It would typically reside in the living room or listening room, to drive each monitor and audio system. The prototype for the DHEC is a multimedia PC. However, the DHEC must be designed for consumer buying habits and sensibilities. It should have the following features:

• 
  Ethernet, DVD/CD, (quiet) hard drive, and no fan (or sufficiently configured for reduced acoustic noise).
  
• 
  DVI output providing at least 1024 x 768 and 1280 x 720 progressive scan at 60 Hz.
  
• 
  Dolby surround sound to drive an amplified six speaker system.
  
• 
  TV broadcast or Open Cable input with video tuner, radio tuner, and MPEG-2/4 encoders.
  
• 
  Optional tuners to connect to proprietary cable and satellite systems.
  
• 
  IEEE 1394 input for video camera.
  
• 
  USB for security and IP protection devices; videoconferencing camera and microphone
  
• 
  Great software – at minimum, it must support PVR and a jukebox for the owner’s audio collection.
  
• 
  Any remote can control generic functions: volume, channel, mute, pause, rewind, etc. Full functional control would either be via a wireless keyboard with on screen picture-in-picture GUI, or a tablet-PC.
  

Note that a tablet is simply a networked device running the control software. Any networked PC could be used to control the DHEC, opening up interesting possibilities, such as browsing the Web from a PC for TV programming, finding an interesting show, and relaying a command to the DHEC to record it. Note, also, that once home networks are fast enough and media servers are attached the hard drive will be optional.

The DHEC provides a considerable challenge from many points of view. It needs to fit Consumer Electronic buying habits and perceptions for an industry that provides total modularity, compatibility and incremental upgrade ability over decades. Most consumer electronics manufacturers are unfamiliar with digital networks and computers in general. The time scale for change in standards is measured in decades. Most likely the change will have to come from the PC industry that is equally unfamiliar with television standards.

Figure 3: Existing home entertainment center

Figure 4: The Digital Home Entertainment Center (DHEC) simplifies and adds function (note: legacy audio cassette tapes assumed to be recorded digitally into the DHEC).

3.1Displays – Beyond HDTV

At first glance it would seem obvious that the 16:9 format would be preferred over the 4:3 format of TV and most monitors. However, from a true cost perspective, 4:3 provides more pixels per unit cost³ and appears to be better suited to computer use. Thus, 16:9 is not inevitable.

Whether current and future TV sets can be used, depends on how one may use them. Some of the major viewing categories are:

• 
  Ordinary TV content.
  TV sets can go beyond current TV content. S-video and new HDTV set quality is adequate for computer art, photos, home video, and DVDs – “TV screen savers”. (see Figure 2.)
  
• 
  Lectures with demos and slides.
  Current television sets are marginal, but can be used if the slides use very large fonts. For most users, a progressive scan monitor with at least SGA resolution (800x600) is required (see Figure 3) unless content is appropriately scaled.
  
• 
  “Mini-mail” and “Mini-browsing” using limited and/or translated content.
  Current TVs can do mail and Web-browsing, but large fonts and the right colors are required, resulting in only a few (10-20) lines of text being displayed at a time. This is a very different and limited experience from full resolution mail and browsing found on PCs today. Web pages usually must be authored specifically for TV sets. However, mini-mail and mini-browsing are expected to be widely used in small portable devices, and we also expect some usefulness in them appearing on TV sets. WebTV has over a million users that view color and size translated Web pages for better viewing on TV sets. A TV format seems to be more readily accepted in Europe, perhaps because of their experience with TV-based Teletext that uses appropriate fonts for example to display a schedule.
  
• 
  Browsing and mail at distance using very large displays.
  e.g. reading the “Times” in bed, or browsing from the living room couch.
  SGA (800x600) is the minimum, useful resolution. Selker (2000) has demonstrated projectors that display on the bedroom ceiling that could be used in a variety of way ranging from reading a newspaper to a planetarium. XGA (1024x768) is desirable for mail and most applications. It is unclear whether HDTV sets that barely meet SGA capability will be adequate based on user expectations.
  

We believe future displays must have monitor characteristics, including progressive scanning and higher resolution. Unfortunately, the television networks and set manufacturers seem slow to catch on, being hampered with 50 years of experience with interlaced scanning. On the other hand, TV manufacturers envision the TV set being the center of the home multimedia. Currently, interlaced displays are cheaper than progressive displays. However, this need not be the case, it is almost entirely a matter of the large difference in volume levels between television sets and computer monitors, together with the channels of distribution.

Some colleagues believe the ultimate device is high-resolution glasses that are capable of accepting all formats, but these are yet to become available. For the foreseeable future, there are limited options available at any price for viewing both television and computer content in what we envision is the true future. A very interesting alternative is the use of computer display DLT or LCD projectors. For home use, these are limited because they are often large, noisy, and need a darkened room. Plasma panels of 30-50” provide one of the nicest alternatives because they are very bright and can be useful for displaying art. Unfortunately, their use is limited by their high cost and power consumption (over $10,000 cost and 350 watts). They are unlikely the monitor for many users.

3.2Building the network: old wires, new wires or wireless?

• 
  data being routed to/from the Internet (up to 5 Mbps);
  
• 
  three video streams (DVD quality: 24 Mbps; HDTV quality: 60 Mbps), and
  
• 
  in-home PC server traffic, which can be almost any amount for backup/replication, video from digital TV tuners to servers, and video intercom/surveillance.
  

If new CAT5 wiring is being added to a home, it is prudent to also include RG-6 co-axial cable for legacy, analog CATV. A single bundle with two CAT5 and two RG-6 cables is common and cheap (installation costs make the cabling cost insignificant). To really hedge your bets, a fiber optic pair can be added to the bundle for little cost.⁴

Ironically, an in-home CATV distribution network could be enhanced to provide a home network without the need for another data network or re-wiring. Such a network would work similarly to the network cable companies already deploy to serve IP to subscribers. Cable networks have already been modified to carry “digital TV” channels. In a similar way, some channels could be utilized in the home for IP networking. Cable is less prevalent in many parts of the world, so clearly it is not a universal solution. For example, Japan is committing to wire homes with fiber that would allow a completely centralized architecture for all types of content distribution. Various European countries use a mix of broadcast, cable, and satellites for distributing what amounts to a small number of TV channels.

Figure 7 shows the “wiring closet” for a home with wired and wireless telephony, IP networking for three rooms, and an 802.11b wireless Ethernet. The closet also has cable and DSL modems for Internet service⁵, a battery backup for brown-outs, a firewall appliance, an Ethernet switch, and a patch panel. Not shown is the coax for TVs in four rooms. If you are not impressed with how compact and tidy this is, go look under your computer desk!

4Content distribution, storage and management

4.1Distribution and content caching for non-real time use

While real time viewing dominates today, we believe this will change. The experience of the Personal Video digital Recorder (PVR) users of ReplayTV, TiVo, and UltimateTV is they begin to watch most content in a time-shifted manner: “prime time no longer exists”. Thus, as long as a good way is found to stuff your media cache with content that you want, real time distribution will only be needed for a few categories like news, sports, and communication (videophone). Cache stuffing may even be the way to go for news and sports. We do not get most news instantly. It comes on the “6 o’clock news” or in the “news cycle” of the all-news station. Viewers may often prefer a cached version of the last news cast to waiting for the next one. Clearly the 6 o’clock news being watched at 6:15 is much better because it permits skipping over the 15 minutes of commercials that accompany the news. Certainly sports highlights can be cached, and sports occurring in different time zones may be inconvenient to watch live.⁶

Almost any distribution network is suitable for cache stuffing. Existing web dialup connections could use the usual unicast methods like FTP and HTTP, or could receive a multicast. Napster-style sharing can be applied to video as well as audio. Existing television channels could be re-purposed for cache stuffing. A single analog TV channel can be used to transmit 4-6 digital streams of VHS quality, or 2 at HDTV quality. We could see some analog channels switch to digital streams at night to stuff caches. Of course, there already is digital TV distribution. However, when cache stuffing is the primary goal rather than real time viewing, we may see transmissions altered to serve that purpose, e.g. send slightly slower than real time in order to add more error correction.

One problem content distributors have with time-shifting is the viewer’s ability to skip the commercials that pay for the content/distribution. However, as digital rights management (DRM – see below) matures, cache stuffing will allow for some interesting opportunities in ad-insertion and pay-per-view that can please both viewers and distributors.

The answers to questions about electronic media distribution will unfold as the technology evolves to deliver content digitally at higher speeds using both the cable TV and digital telephony infrastructures. There will always be some content (news and sports) that will be transmitted live via TV, radio or webcast. However, we believe that time-shifted viewing will be dominant, and most distribution will take the form of a file transfer rather than a real time stream.

In contrast to video, high-quality streaming audio is attainable over current dialup connections, and non-real-time distribution is common, the most famous example being Napster. An estimated 25,000 thousand radio stations worldwide unicast their content via the Web to reach a completely worldwide audience.⁷ However, nearly all are only available in real time, creating barriers to listeners in different time zones. We expect that, someday, worldwide radio programs will be cached at least as much as they are streamed. As with TV, distribution methods that today are intended for real-time consumption will become cache-stuffing methods (e.g. the digital satellite radio networks now being launched).

4.2Location: will personal content be stored inside the home or outside with a service?

The “right” place to store personal content is hotly debated. Personal content could be stored on a central service, a home server (or servers), or on one or more home DHECs associated with each viewing station (like today’s PVRs). Some observers believe that all content will be stored away from the home by service providers that maintain very large servers (hubs). Mail services such as Hotmail and Yahoo are examples of a central approach. There are embryonic examples of this trend for photos like MSN, Ofoto and Shutterfly. We see serious problems with a central service approach, including security, control, lack of bandwidth (for the foreseeable future), and most importantly, the cost and commitment to maintain someone’s personal files forever.

Terabyte personal stores for PCs at negligible cost are likely by 2005. These will greatly reduce the incentive to store outside the home and outside one’s control and ability to retrieve content such as letters, photos and video forever. We believe that personal content will be stored primarily in the home, and will be stored outside the home for the purposes of backup and serving to or sharing with others. Such a personal store presents its own challenges (Bell, 2001).

As example of the lack of incentive to store outside the home, consider the disk-based personal video recorders (PVR). These recorders capture content from CATV, store it on embedded disks in digital format, and then replay the content on the attached TV set. A few TV set manufacturers have built in the PVR capability. It is vastly cheaper to have these disks in the home than to buy the necessary bandwidth to view the same TV programs as video on demand.

The “centrality” of home servers versus distributed storage is less important from an architecture viewpoint than whether content is stored external to the home and is available as a service as discussed above. For example, it may be that every DHEC in the home is a server. Alternately, there could be a dedicated media-server-appliance. Either or both can exist, so long as the Home Media Network can carry any A/V source to any listening/viewing station sink. However, in our model of the future, all content will be distributed to the home and reside on home servers and be distributed on the home IP network—not as analog audio or video.

4.3Content protection

Intellectual property issues are currently holding up adoption of time-shifting. For example, Internet radio stations do not allow time-shifting, for fear their commercials will be skipped, and because in the USA time-shifting puts it in a different distribution category, requiring increased royalty payments to song publishers. Early PVRs lack the ability to remove the digitally stored content because this ability might allow or facilitate copying. In the case of UltimateTV, the PVR functions are integrated with a set top box that selects and the channel, including the electronic program guide (EPG) in order to manage program recording. By integrating the two functions, then no interfaces are introduced that would allow digital copying. Going even further, PVRs and DVD players ensure that their analog output is impossible for most VCRs to tune (and VCR manufacturers are “good citizens” by ensuring their tuners are not robust as those of TVs).

Whether these restrictions will continue is unclear. It is difficult to speculate that digital content will not be able to be freely transmitted among components. However, intellectual property will have to be protected from all known forms of theftware, so the development of digital rights management (DRM) is critical for the next generation of media distribution to arrive. Sadly, sorting out DRM will involve legal and politic issues, which may delay progress. On the technical front, the general flavor of content protection to date is encrypted content and authenticated devices. Standards like HDCP 1.0 (High-bandwidth Digital Content Protection) are being introduced.

The danger of content protection schemes is that they may also protect the status quo, and keep us from realizing some of the exciting new prospects of the Home Media Network. For example, one way to protect video content is to have a tuner device that tunes and decodes an encrypted stream and overlays it directly on the video output. This makes manipulation or analysis of the content is impossible, preventing the key value additions of computerization. In contrast, SCM Microsystems www.scmmicro.com produces tuners that decode premium content and present raw MPEG to a PC. One might think that products like this let the cat out of the bag for content producers, but, in fact, licensing can be used to stop such products if they are seen as a threat.

In a few years, we may all look back at this time as the end of an era when so much content (TV & radio) could freely and legally be recorded for personal use. Devices should come to market quickly to enable a feeding frenzy on this “all you can eat” content.

5Conclusions

As we wait for digital media appliances to emerge, the interim will see “digital transformers” that allow existing analog media devices such as TVs and stereos to connect to the network. Digital audio transformers are already on the market. The newly introduced tablet PCs have the potential to be very useful as digital transformers, universal remote controls, personal TV sets, books, and plain old networked computers

The speed at which the vision of the Home Media Network materializes is entirely up to the collective will of the hardware and software manufacturers from the computer and consumer electronics industries. The consumer electronics industry must give up on analog – including cable TV, TV sets, and analog audio. The computer industry must provide an alternative to the monolithic PC in the form of the digital home entertainment center (DHEC) we have outlined. They must also allow for the “disintegration” of the PC, where all its I/O peripherals, and especially media peripherals, become smart, but limited function, networked devices.

For displays, we need more than HDTV. Displays must be progressive scan and high resolution. By adopting such a standard, displays can be produced as cheaply as existing interlace standards. Some manufacturers may build all functions into their TV sets versus using a component model that separates display, tuner (broadcast, cable, or satellite), recording, and computer access.

The home network must be IP-based and capable of supporting 50-100 Mbps. CAT5 wring is a proven approach, and it appears that other technologies are coming up to speed. The Home Media Network can support both central server “multi-media home mainframe” architecture and a fully distributed A/V (source)- listening/viewing station (sink) architecture. It is likely that cable TV will continue to exist and evolve to be significantly more open and digital friendly.

The lineup for home media networking is in place: cheap, high-capacity hard drives, excellent compression and networking standards, and a host of prototypical experience in the form of PCs. The thorniest issue is probably Digital Rights Management, but progress is being made. It is clear that now is the time for industry to make the Home Media Network a reality.

Media Applications (sidebar) (near section 1?)

Analog Distribution (sidebar)

The connection between a computer and amplifier-speakers, TV set, or video monitor is analog. The distance between digital source and A/V equipment is nominally a few feet. There are ways to extend this distance up to a few thousand feet. For example, 3 of the 4 pairs in a CAT5 cable carrying low impedance signals and using transformers at each end can be used to drive a remote speaker and monitor. Additionally, a computer with a composite (NTSC) output can have this signal modulated to an unused TV channel and mixed into the RG-6 coax home cable wiring, so that the PC becomes the source of a TV channel to the home TVs. Unfortunately, some CATV set top boxes filter out channels not part of their distribution network.

When the distance is extended between the computer and A/V stations, it is usually necessary to have a “remote” of some sort. Various radio linked mice and keyboards can fulfill this function. Alternatively, as described above, a PC at each listening or viewing station solves both the transformation to analog and the user interface problem.

A CyberAll Media Server Example (sidebar)

Patrick Schmitz at BARC built an experimental video server with a wide range of PVR capabilities. It includes being able to display the media e.g. art, slides, home video, train rides, created in the Cyber All Project (Bell, 2001). The first phase of the project is aimed at understanding the various encoding subsystems, distributing content via standard LANs (including 802.11b wireless Ethernet), replaying content on either computers and TV sets, control, and quality of service issues. The result of this has been to demonstrate that all of the current standards can be played using today’s wired networks. Even 802.11b is sufficient for everything but the highest resolution HDTV.

The second phase will concentrate on features that the PC and web access enable.

Control is from any client via a browser interface and includes the standard functions for channel surfing, recording, playing, etc. For TV viewing, a relatively generic remote suffices.

Acknowledgements

References

Bell, G. "A Personal Digital Store", Communications of the ACM, Vol. 44, No. 1, January 2001, p86-91.

Kurzeil.AI.net