Testing on real phones: why it matters and how

What a simulation leaves out

The Chrome DevTools documentation is plain about this. Device Mode is a first-order approximation of how a page looks and feels on a mobile device. You do not run your code on a mobile device. You simulate the experience from your laptop or desktop. The documentation says that some aspects of mobile devices can never be simulated, for example that the architecture of mobile processors is very different from that of laptop and desktop processors. When in doubt, it says, run your page on a mobile device.

Lighthouse works differently but has a similar limit. In its default mobile test (Lighthouse 13.5), it emulates a Moto G Power (2022)-sized screen, 412 by 823 pixels at a pixel ratio of 1.75, and it uses a simulated slow 4G connection with a slowdown of the processor. It does not slow anything down while it runs. It loads the page once and calculates what a slower device would have seen. That makes results fast and fairly repeatable, but it is a model.

What these models cannot capture:

• The processor and memory. A phone runs JavaScript more slowly than a laptop, and has less memory to hold a heavy page.
• Heat and battery. A hot or low-power phone slows itself down.
• Other browsers. A phone has its own browser. When web.dev wrote its guide to the back/forward cache (updated in 2023), iOS required every browser to use Safari's WebKit rendering engine, so iPhones behaved like Safari whichever browser app they ran. Rules can change, so check which engine your visitors' browsers use.
• The real network. Mobile data varies from second to second, between rooms and between operators.
• Other things running. Background apps, the keyboard, and the way a visitor really taps and scrolls.

What to test, and on which phones

You do not need a drawer of phones. Choose a few that represent your visitors:

1. Look at your visitors. Your analytics show which devices and browsers they use. The Chrome UX Report shows your real-visitor numbers, and in PageSpeed Insights the field data section is the part from real phones and computers. See the Chrome UX Report.
2. Pick a typical phone and a slower one. One mid-range Android phone, one older or low-end one if your visitors use them, and one iPhone.
3. Test your key pages and flows. The home page, a landing page, a product or service page, and the path to a form or purchase. Include the actions: opening the menu, filtering a list, adding to the cart.
4. Test on the network your visitors have. Try mobile data, not only office Wi-Fi.

Inspect and record a real Android phone

Chrome can drive a real Android phone from your computer. The DevTools documentation describes the steps:

1. On the phone, open Developer Options and turn on USB debugging.
2. On your computer, open Chrome and go to chrome://inspect#devices. Make sure Discover USB devices is checked.
3. Connect the phone with a USB cable. The first time, accept the debugging prompt on the phone. When the phone's model name appears, the connection works.
4. Open Chrome on the phone. Its tabs appear in the list. Enter a URL in the Open tab with url box to open a page on the phone, then click Inspect next to it.
5. A full DevTools window opens on your computer, connected to the page on the phone.

From there you can use the Network and Performance panels as usual, and the work is done by the phone's own processor and network. The documentation adds some points:

• The version of Chrome on the phone decides the version of DevTools that opens, so an old phone may show an older-looking DevTools.
• You can screencast the phone's screen into DevTools. Screencasts lower the frame rate, so turn them off while you measure scrolling or animations.
• If the device is not found, try a direct USB connection instead of a hub, unlock the screens, and try another cable.

A Performance recording from the phone shows how long the long tasks really are on that hardware. Compare it with your laptop's recording. See the Performance panel and the Network panel.

iPhones

The Chrome documentation we read covers Android only. For an iPhone, Apple's Safari has its own web inspector, which needs a Mac. Check Apple's current documentation for the steps. Even without it, you can still load your key pages on an iPhone and use them: scroll, tap menus, fill a form, and see where it hangs.

Without your own devices

If you cannot get devices, some online testing services offer real phones in a lab, and WebPageTest lists Android and iOS as browser choices in its quick start guide, alongside emulated devices that use Device Mode. Their availability and prices change, so check them before you rely on them.

Compare the simulated and the real

Run the same page three ways and write the results down:

If the real phone is much worse than the simulation, look for work that the simulation understates: heavy JavaScript, large images that the phone must decode, many third-party scripts. If it is much better, your visitors may have more capable phones than the simulation assumes. Real-visitor data settles the question for your site.

Mistakes to avoid

• Treating a mobile score as a phone result. It is a model.
• Testing only on your own, new phone. Your visitors may have older ones.
• Testing only on Wi-Fi.
• Leaving the screencast on while you measure. It lowers the frame rate.
• Testing once. Repeat, and compare the pattern.

Check your own page

Run the free audit for a simulated mobile result. Then load the same page on a real phone on mobile data, tap through it as a visitor would, and note where it waits. Where the two disagree, trust the phone.

Sources

• Simulate mobile devices with Device Mode and Remote debug Android devices in the Chrome DevTools documentation: the limits of simulation, and the steps to inspect a real phone.
• Lighthouse in Chrome DevTools: simulated throttling does not slow the page down while it runs.
• Lighthouse configuration source, version 13.5: the default mobile screen, device name and simulated throttling.
• Back/forward cache on web.dev: the note that all browsers on iOS use WebKit.
• Quick start guide in the WebPageTest documentation: the browsers and emulated devices it offers.