A lot has changed since our first evaluation of Wi-Fi products eight months ago. Many older products have been upgraded to full compliance with the 802.11g standard. Lots of new products have come to market, some with Wi-Fi certification. With the products stabilizing, we decided it was a good time to resume our testing and to report on all three versions of Wi-Fi.
"Wireless Is Magic" -- Our Evaluation of 802.11g Wi-Fi (BBHR June 2003) described our first Wi-Fi evaluation. That article reported on our tests of four then-new 802.11g devices: two access points and two PC card notebook adaptors, all based on the same Broadcom chip. We measured throughput at nineteen locations around our house while transferring large files between PCs.
802.11g is the newest "flavor" of Wi-Fi. At the time of our earlier tests, the standard had just been approved, and all the tested devices were based on earlier drafts of the standard; none had completed Wi-Fi Alliance certification testing. Now nearly all devices comply with the final standard, and some have passed Wi-Fi tests.
This report covers the first part of our Round Two testing: "baseline" measurements with a wireless notebook adaptor plugged into a notebook PC in the same room as the access point. This provides a good indication of the maximum throughput possible with each combination of an access point and a notebook adaptor. The test results are summarized below; the detailed results are reported in the Round Two Wi-Fi Baseline Tests section of our website.
In an upcoming issue we will report how the throughput is degraded by distance, walls, floors and other obstacles.
Three "Flavors" of Wi-Fi
The "11a" and "11b" standards were both published in 1999, while "11g" was published in 2003. 11g products are all back-compatible with 11b. 11a runs in a completely different frequency band so "11a-only" products won't work with 11b or 11g.
The Wi-Fi Alliance has tried to clarify this with its logo program, but it can be difficult for consumers to tell the products apart. The picture shows a Wi-Fi logo for a product which supports all three versions and has been certified for WPA (a new security specification). Some "combo" products support multiple standards but have only been certified for one.
Our Baseline Tests
We are conducting an extensive series of tests involving five access points and six notebook adapters, covering all three flavors of 802.11. Most of the tested equipment was loaned to us by Linksys and SMC Networks.
When completed, these tests will include three sets of measurements:
We have completed the baseline measurements and summarize them below. We expect to complete the other two test series soon and report on them in an upcoming issue.
Physical rate and throughput
The quoted speeds of 11 Mbps (802.11b) and 54 Mbps (802.11a and 11g) are maximum speeds, representing the maximum bit rate through the air (this is sometimes called the "physical" or "PHY" rate). The maximum rate at which data can be transferred from one computer to another -- called the "throughput" -- would be much more useful, but is not published. Our "baseline" tests were intended to estimate the maximum throughput for each version of Wi-Fi.
Our throughput measurements are quite a lot lower than the quoted physical rate; we calculated the "efficiency" of these products at between 35% and 53%. This did not surprise us, since the 802.11 protocol includes a lot of "overhead" -- airtime consumed by "handshaking" rather than useful data.
Updating firmware and drivers
Although we knew that the vendors had released updated firmware and drivers for many of the devices we were testing, we chose to run a preliminary series of baseline tests with the devices as shipped from the manufacturer. It did not surprise us that some "draft 11g" devices from one vendor did not work properly with "draft 11g" devices from another.
We then tried to update all of the devices, installing new firmware for the access points and new drivers for the notebook adapters. We found this to be a rather difficult process, requiring several steps:
We succeeded in upgrading all of the access points and most of the PC cards. After updating firmware and drivers, all 11g devices--even from different manufacturers--worked properly together.
Since we sometimes found better results after updating firmware and drivers, we recommend upgrading. This is particularly important for 802.11g devices, many of which were shipped with "pre-standard" firmware or drivers.
Several of our test runs graphically illustrate the improvement from upgrading firmware.
As an example, we tested the SMC2804WBR router with an SMC2336WAG PC card first with the original router firmware as shipped from the factory and then with the latest firmware after an upgrade.
The "run chart" on the right shows time for each run with the original firmware as shipped from the factory.
The results appear quite "jittery". The throughput is about 14.8 Mbps.
This shows the same router and PC card after upgrading the router firmware.
The difference is quite dramatic. The throughput is now 19.0 Mbps, an improvement of more than 25%.
In addition, most of the new 802.11 devices support an improved form of security known as "WPA", which is considerably more secure than the WEP security in the original Wi-Fi. Many of the upgrades add WPA support for older devices.
While new production products include support for the final 802.11g standard and WPA, earlier-production products may still be in inventory. These will need upgraded firmware or drivers.
Baseline test results
As in our earlier tests, we measured the throughput while transferring large files between PCs. Unlike our earlier tests at nineteen locations around the house, we made all these baseline measurements with our notebook PC close to the access point in the same room. This provides a good indication of the maximum throughput possible with each combination of an access point and a PC card notebook adapter.
As before, we did these tests in only one house, with one sample of each product and with one particular test tool of our own design. We have no way of knowing how well other samples and other products in other homes will perform.
Our test results are reported in detail in the Home Networking - Wi-Fi Evaluation section of our website. The Round Two Wi-Fi Baseline Tests page includes the measured throughput for each combination of an access point and a network adapter, with many run illustrated by charts.
Here's a summary of what we found:
These tests were intended to provide a realistic estimate of Wi-Fi throughput with wireless devices close to access points. The maximum speed of 802.11b is about 5 Mbps, while 802.11g and 802.11a are close to 20 Mbps, about four times faster.
More recent products -- access points and PC cards -- performed better than older ones when running 802.11b. This would indicate that wireless product designers and manufacturers are improving their skills at realizing the potential of the standards.
Many 11b/11g and 11a/11b/11g consumer devices are now on the market. The price difference between these devices and 11b-only devices is small, so we believe consumers should avoid 11b-only devices and favor those with 11b/11g and 11a/11b/11g.