Author: Xuangui@DuncanSecTeam

0x00: Preface

China Telecom is the largest fixed-line broadband access service provider among the three major operators and is also the earliest operator to start building fiber optic broadband. As shown in the figure below, from the early e8-C to the later Tianyi Smart Gateway 1.0, 2.0, 3.0, and the latest 4.0, its fixed-line fiber optic terminal access equipment (hereinafter referred to as: optical cat) has always been at the forefront of the three major operators. We hope to introduce the Telecom optical cat in depth and detail through a series of articles, focusing on the technical details of network security involved.

0x01: Article content

In the first two articles, we analyzed the source code acquisition, boot process, and web application of the Telecom Optical Cat 3.0 device. Based on these analyses, we can better control or analyze the optical cat system. In this article, we will continue to analyze the optical cat, but with a focus on the LXC container running on the optical cat. Due to limited capabilities and very limited reference materials, please feel free to point out any inaccuracies as mentioned in the text!

0x02: lxc container

2.1 What is an LXC container?

LXC was first introduced in 2008, adopting most of its features from its predecessors, Solaris Containers (or Solaris Zones) and FreeBSD jail. Containers decouple software applications from the operating system, providing users with a clean and minimal Linux environment while running all other content in one or more isolated 'containers'. The purpose of containers is to launch a set of limited applications or services (usually referred to as microservices) and run them in an independent sandbox environment. This isolation prevents processes running in one container from monitoring or affecting processes running in another container. Similarly, these containerized services do not affect or interfere with the host. The idea of integrating many services scattered across multiple physical servers into one is one of the many reasons why data centers choose to adopt this technology.

The concept of container technology, initially known as FreeBSD jail in 2000, was used to partition the FreeBSD system into multiple subsystems (also known as Jails). Jails were developed as a secure environment, allowing system administrators to share these Jails with multiple users within or outside the enterprise. In 2001, through Jacques Gélinas' VServer project, the implementation of isolated environments entered the Linux domain. After completing this foundational work for multiple controlled user spaces in Linux, Linux containers began to take shape and eventually evolved into the current form. Soon, more technologies were integrated, turning this isolation method from an idea into a reality. Control groups (cgroups) is a kernel feature that can control and limit the resource usage of a process or a group of processes. The systemd initialization system can set user space and manage their processes, using this system to more strictly control these isolated processes. Both of these technologies, while increasing overall control over Linux, have also become important frameworks for maintaining environmental isolation.

2.2 General Process of Using lxc Containers

The following figure shows the general process of using lxc containers, mainly including: installing lxc services, checking the lxc running environment, running lxc containers, and terminating lxc containers.

0x03: Analysis of China Telecom Optical Router 3.0 LXC Container

Based on the preliminary analysis and the practice of running various plugins inside the OpenWrt container using telnet, here we present a conjecture of our team regarding the LXC container deployment of the China Telecom Fiber Optic Router:

1. The China Telecom Fiber Optic Router uses LXC to deploy the OpenWrt Privileged container to run the complete OpenWrt system;

2. Using the OpenWrt container, the China Telecom Fiber Optic Router can directly run OpenWrt plugins and intelligent plugins developed based on the OpenWrt framework.

Next, we will conduct a series of analyses and experiments to test the previous conjecture.

3.1 OpenWrt Container

When logging into the telecom optical router, it is often possible to see a URL address ending with “/cgi-bin/luci” similar to the image below. It is this address that leads us to suspect that the telecom optical router 3.0 may have been rewritten based on OpenWrt to support the new application scenarios mentioned in the previous article [3, 4]. Friends who are familiar with firmware flashing and OpenWrt should know about the powerful software ecosystem and rich third-party plugins of OpenWrt. In order to gain a deeper understanding of the telecom optical router 3.0, clarify the internal implementation mechanism, and better operate this generation of optical router, our team has delved into this interesting research field.

We tried to find the connection between the optical network terminal and OpenWrt through search engines, and there is not much information available. Among them, [5] introduces how to access the internal OpenWrt of the optical network terminal through Dropbear SSH, but the model of the optical network terminal mentioned is not from the same manufacturer as the one we have. [6] provides a more detailed introduction to the LXC container in the optical network terminal of the telecommunications company, and analyzes the saf program used to apply containers. This article is particularly helpful to our team, and we would like to express our gratitude to songee@chinadsl.net.

Combining the analysis of [6], we can view the parameters passed in by the saf program when executed through the ps command, and judge that the parameters 8, 9, and 10 are probably hard-coded.

In this way, we can take 'saf service 8 9 10' as the target string and search for it in the script and binary elf files (the import, export table, and resource section of the elf file are all in plain text) of the optical network terminal. We found that the string exists in the /bin/proxyDaemon file. Following the trail, we found that /bin/reg_server and /bin/startup programs call proxyDaemon. Among them, /bin/reg_server should be called when the optical network terminal is not registered, while /bin/startup is called in the normal startup process of the optical network terminal. Further tracing back, the /etc/init.d/rc32 script ultimately calls /bin/startup, and the script code is as shown in the figure below:

3.2, saf program

The article [6] provides a simple analysis of the saf program, considering that the saf program is responsible for calling lxc to create, run, and monitor OpenWrt containers, but the information provided is relatively limited. We will extend appropriately based on the article [6], trying to restore a more detailed OpenWrt container.

As mentioned in the previous text, the /etc/init.d/rc32 script file executes the /bin/startup program in a background running mode and passes in the parameters service 8 9 10, etc., a total of 4 parameters. By looking at the /proc/mtd file, 8, 9, and 10 correspond to the following 3 storage areas in the FLASH: framework1, framework2, and apps.

再结合saf程序的帮助信息，如下图。我们基本可以判断saf会根据指定的mtdAnd combinedOpenWrtProgram help information, as shown in the figure below. We can basically judge3Will be based on the specified

mtdNumber, load the specified file, and then create and start the container. So the problem is, run aLXCContainer, why do you need to load

How to search for the files in the 3 storage areas?3.3And useLXCProvided software to operate the container in the optical network terminal/usr/binDirectory, we foundLXCContainer management required programs, as shown in the figure below. In order to better analyze the container in the optical network terminal, we foundOpenWrtInstalledLXCIn the environment, and tried to deployOpenWrtContainer. So, we tried to use the existing

Container operation program, viewContainer information.Variables can be executed successfully. As shown in the figure below, byLxc-ls

Programs can view the containers running in the current system, but unfortunately the telecommunications optical network terminal has been customized,Programs cannot be run directly.So, we tried to run in the writable/tmpDirectory to createOpenWrtbinVariables can be executed successfully. As shown in the figure below, byDirectory, and fromThe required program, and copy the/tmp/binAdded toPATHVariables can be executed successfully. As shown in the figure below, bylxc-lssafOnly one container exists in the current system, that is. This experimental result is consistent with our previous guess, that is, the Tianyi gateway3.0LXCThroughOpenWrtSystem. But the telecommunications optical network terminal has deployedLXCApplications are limited to this? Why run aOpenWrtSystem,safNeeds to be loaded fromFLASHStorage loaded3How to search for the content of the 3 storage areas?

3.4Matryoshka

We have alreadydumpIn the runtime file system of the followinglxcWhen searching for related programs, we found an interesting phenomenon, which once confused us, as shown in the figure below. Used to startLXCContainerlxc-startPrograms actually exist simultaneously in the host file system of the optical network terminal andOpenWrtContainer file system (OpenWrtThe corresponding file of the container in the optical network terminal is located/opt/upt/framework/rootfs/usr/binDirectory).

As shown in the figure above, we see such search results, the first reaction is the OpenWrt built-in LXC software? But, compared with the OpenWrt container deployed in Ubuntu, we can determine that OpenWrt does not natively support LXC, that is, the original OpenWrt is not installed with LXC. Therefore, we further examined other programs related to LXC in the /opt/upt/framework/rootfs/usr/bin directory, as shown below. Surprisingly, LXC is indeed installed in the OpenWrt container, but the supporting programs have been reduced to 4. So, we can draw two conclusions:

1. The OpenWrt system also uses LXC containers;

2. Containers in the OpenWrt system only need to perform simple start, stop, and other operations.

In order to verify the above conclusions, we tried to copy the host file system of the optical network terminalLXCSupporting software is copied toOpenWrtIn the container, we finally see the software running in the optical network terminalLXCContainer.

As shown in the figure above, we can see thatOpenWrtThe system also runs6AnLXCContainer. Of course, why the operator will adopt this Russian doll-like way to run its own intelligent plug-ins (let's call them that for the sake of convenience), how to run and manage plug-ins, these contents belong to the operator's business field, and we will not elaborate on them.

0x04: Summary

This article is the follow-up and the last article of the security analysis of China Telecom's Tianyi Gateway 3.0. Due to time constraints, the analysis of the most common optical network terminal (Home Gateway Unit, HGU) in the operator's fiber network terminal (Optical Network Terminal, ONU)【6】has come to an end. From the perspective of traditional network security and Internet of Things network security, the attack surface of China Telecom's optical network terminal mainly focuses on the following aspects (the security of the operator's ITMS management system is not considered):

1. Web applications of optical network terminal, such as asp scripts, cgi programs, etc.

2. Optical network terminal wireless network access, most 3.0 optical network terminals come with Wi-Fi and are open by default.

3. OpenWrt and its attack surface, can attackers use OpenWrt to achieve persistence? Will the plugins supported by OpenWrt such as tcpdump be used for network attacks?

Please forgive any inaccuracies in the text!

0x05: Reference

【1】In the era of cloud-native, do you really understand the difference between Docker and LXC? https://zhuanlan.zhihu.com/p/411276481

【2】What is Linux Container. https://www.redhat.com/zh/topics/containers/whats-a-linux-container#What-is-the-LXC-project?

【3】MWC ended! China Telecom's 'Tianyi Gateway' 2.0 was perfectly presented!//https://www.sohu.com/a/153967240_515599

【4】Create a new ecological connection for smart homes, China Telecom will launch the 3.0 version of Tianyi Gateway next year//https://www.sohu.com/a/208951906_234937

Wake up the hidden universe of Huawei smart gateway modem: OpenWRT. https://www.right.com.cn/forum/thread-352805-1-1.html

What are the differences between ONU, ONT, SFU, HGU...? https://zhuanlan.zhihu.com/p/402225304