International Journal of Technology | 2026
Authors: Nishi H.; Yano S.; Miyashita M.; Onishi S.; Goto Y.; Kondo T.
DOI: 10.14716/ijtech.v17i1.8195
Journal: International Journal of Technology
Year: 2026
Publisher: Faculty of Engineering, Universitas Indonesia
Document Type: Article
Open Access: All Open Access; Gold Open Access
Cited by: 1
Federated learning (FL) has emerged as a key paradigm for decentralized data privacy-preserving machine learning. However, substantial communication costs often hinder its practical application, especially as deep learning models scale to millions or billions of parameters. This communication bottleneck becomes particularly acute in heterogeneous networks with clients who are resource-constrained. To address this challenge, this study proposes a novel FL framework that leverages black-box optimization, specifically the zeroth-order (ZO) method, to reduce communication overhead. The proposed method, named ZO-FedSGD, reframes the learning process to eliminate the need for transmitting high-dimensional model parameters. Instead, each communication round involves exchanging only a constant number of scalar values, including a random seed and function evaluations, making the communication cost independent of the model size. Extensive experiments were conducted to compare ZO-FedSGD with the existing FedAvg algorithm on the MNIST datasets. The evaluation focused on model accuracy and total communication efficiency. Our results reveal a trade-off: ZO-FedSGD required more rounds to converge and achieved a slightly lower final accuracy. However, it demonstrated superior communication efficiency—to reach 90% accuracy, ZO-FedSGD required approximately 104 communicated parameters, compared to 106 for FedAvg, representing a two-order-of-magnitude reduction. In conclusion, this study validates ZO-FedSGD as a viable and highly efficient alternative for FL in communication-constrained scenarios. It offers a new direction for designing scalable FL systems and a promising solution to the statistical heterogeneity problem. © 2026 Faculty of Engineering, Universitas Indonesia. All rights reserved.
Black-box optimization; Federated learning; Two-point estimation