Google's AI Photo Tool Estimates Body Fat From Selfies

Google Research has unveiled a novel artificial intelligence system called PhotoScan that can estimate body composition, including body fat percentage, by analyzing simple 2D images taken with smartphone cameras. This innovative approach aims to offer more accurate health insights than traditional wearables, potentially even predicting conditions like insulin resistance.

The United States faces a significant public health challenge with obesity, as approximately 73.1% of American adults are classified as overweight or obese, according to the National Institutes of Health. While stepping on a scale offers a basic weight measurement, a comprehensive understanding of body composition, particularly fat distribution, often requires complex methods. Some of the most precise assessments historically rely on dual-energy X-ray absorptiometry (DXA) scans, which utilize a form of X-ray technology.

Google's PhotoScan system was trained using data from DXA scans to learn how to correlate visual cues with body fat levels. The researchers also incorporated data from MRI scans to further refine the model. The crucial step involved training the AI with actual photographs captured by smartphone cameras. The resulting system demonstrated a higher degree of accuracy in estimating overall body fat compared to the bioelectrical impedance analysis (BIA) sensors commonly found in fitness trackers and smartwatches.

Moreover, PhotoScan showed significant accuracy in estimating body fat ratios, such as the abdominal to gluteal (A/G) ratio and visceral to subcutaneous (V/S) fat ratio. These specific measurements are beyond the capabilities of current wearable BIA technology, offering a more detailed view of fat distribution within the body.

Potential Health Applications Beyond Body Fat

Beyond its primary function of assessing body composition, Google Research explored the potential of PhotoScan to predict health conditions. The team investigated whether the estimations derived from the AI tool could help forecast the risk of insulin resistance, a precursor to type 2 diabetes. This aligns with Google's broader interest in leveraging technology for health monitoring, as evidenced by features like the upcoming Insulin Resistance Trends in Health Guardian on the Pixel Watch.

The development of PhotoScan addresses a key limitation in current health tracking: the accuracy and depth of body composition data. Wearables provide convenient, albeit often less precise, daily metrics. PhotoScan, by contrast, offers a path toward more clinically relevant insights accessible through a common device—a smartphone. The ability to accurately estimate visceral fat, for example, is particularly important as it is linked to a higher risk of cardiovascular disease and metabolic disorders. By making such estimations more accessible, Google hopes to empower individuals with better information for potential lifestyle interventions.

While the technology shows great promise, its widespread adoption faces several hurdles. Convincing users to share personal body images, even with stringent privacy policies, presents a significant challenge. Furthermore, even with accurate data, motivating individuals to make the necessary, often difficult, dietary and lifestyle changes remains a complex behavioral issue. Google has not announced any immediate plans to commercialize PhotoScan, but the underlying technology could potentially be integrated into future Google Health offerings, providing a new tool in the pursuit of proactive health management.