Start with the red envelope grabbing plug-in for Android phones
In order to prevent chain theft, this article starts from Yu Guo's blog. Please indicate the source for reprint! Original link: https://www.cnblogs.com/yuxiuyan/p/14524302.html ，
Recently, the mobile Housekeeper on Android phones has updated a new version, providing a red envelope alarm clock function. As long as there is a wechat red envelope or QQ red envelope, it will be automatically reminded. It coincides with the recent work of UI automation, using the UI Automator framework. A few lines of code can make the mobile phone automatically complete some operations. It's very interesting. Let's pick up the principle behind it today.
First, the official document town building: https://developer.android.com/training/testing/ui-automator
In traditional manual testing, we need to click some control elements to see whether the output results meet the expectations. For example, in the login interface, enter the correct user name and password, and click the login button to log in normally.
If these operations need to be performed manually every time, it will require a lot of labor costs. For example, there are tens of thousands of manual use cases for mobile QQ Android. Therefore, we need to vigorously promote automated testing.
As the top layer of the test pyramid, UI automation undertakes the task of end-to-end demand regression and gray level verification, and its importance is self-evident.
UI Automator, as a tool launched by Google for UI automation test, has excellent API and community documents. It is also the mainstream Android automation testing framework. It provides a series of methods for obtaining page control elements and operation elements on the mobile phone, which is very convenient.
Note: the UI Automator test framework is an instrumentation based API that runs on Android junitrunner. At the same time, the UI Automator test only runs on Android 4.3 (API level 18) or above.
Think about our usual process of grabbing red envelopes?
If you are now brushing the play, the notice bar reminds you that there is a red envelope on wechat, so you click the message in the notice bar, enter the wechat page, find the red envelope, click the button to remove the red envelope, shake your hand and get a few cents.
In fact, these steps are completely manual work. If only a robot could grab them automatically!
Behind this robot is accessibility service. Of course, we will talk about its specific role later.
According to our existing logic, automatic red envelope grabbing can be roughly divided into the following steps:
The two most important steps are identification and operation. Next, let's talk about these two steps.
First, let's take a look at the UI Automator viewer, which is located in the < Android SDK > / tools / bin directory. It can easily scan and analyze the currently displayed interface components on Android devices and display a complete control tree and the attributes of a leaf node (control element).
From the above figure, we can see that a login button element on the page has its own text attribute, resource ID attribute, content desc attribute and so on.
In UI Automator, there is a uidevice class. You can view these control elements through the findobject method.
UiObject2 login_btn = uiDevice.findObject(By.desc("登录"));
Now let's go deep into the findobject method,
public UiObject2 findObject(BySelector selector) {
// 这里返回匹配选择器的第一个节点，如果没有找到匹配的话，就返回null
AccessibilityNodeInfo node = ByMatcher.findMatch(this,selector,getWindowRoots());
return node != null ? new UiObject2(this,node) : null;
}
You can see that a selector selector is passed in here, and then query in the findmatch method of bymatcher. If it is found, it returns a node of accessibilitynodeinfo. If it is not found, it returns null.
First, what is bymatcher? This is a utility class. Through its method, we can search for nodes matching the selector in a tree structure.
The findmatch method is very simple. It is a tree search method that starts from the root node. Needless to say.
What is accessibility nodeinfo? This is equivalent to a node. From the perspective of accessibility service, this is an accessible control node.
So, is the third parameter of findmatch the root node of the passed in control tree? Let's take a closer look at the key code of the getwindowroots method here,
/** 这里返回活动窗口容器的root节点的列表 */
AccessibilityNodeInfo[] getWindowRoots() {
// 等待线程空闲后再执行
waitForIdle();
// 初始化一个root节点的集合
Set<AccessibilityNodeInfo> roots = new HashSet();
// 通过UiAutomation获取当前最底部的根窗口容器的root节点
AccessibilityNodeInfo activeRoot = getUiAutomation().getRootInActiveWindow(); // 这里使用UiAutomation的方法
if (activeRoot != null) {
roots.add(activeRoot);
}
// 多窗口容器的搜索
if (UiDevice.API_LEVEL_ACTUAL >= Build.VERSION_CODES.LOLLIPOP) {
for (AccessibilityWindowInfo window : getUiAutomation().getWindows()) { // 这里使用UiAutomation的方法
AccessibilityNodeInfo root = window.getRoot();
…………
roots.add(root);
}
}
return roots.toArray(new AccessibilityNodeInfo[roots.size()]);
}
It should be mentioned here that uiautomation is an automation framework released by Google in Android 4.3. It provides the ability to interact with the underlying system.
Next, let's look at the key code of the getwindows method of uiautomation:
public List<AccessibilityWindowInfo> getWindows() {
……
return AccessibilityInteractionClient.getInstance()
.getWindows(connectionId);
}
Here, the instance of accessibilityinteractionclient is obtained, and then the getwindows method result of the client is returned. Then take a look at the key code of this getwindows method,
public List<AccessibilityWindowInfo> getWindows(int connectionId) {
……
IAccessibilityServiceConnection connection = getConnection(connectionId);
if (connection != null) {
// 首先去查询缓存，如果缓存是有的，直接返回
List<AccessibilityWindowInfo> windows = sAccessibilityCache.getWindows();
……
return windows;
}
……
// 如果上面的缓存不存在，就调用connection.getWindows方法
windows = connection.getWindows();
……
if (windows != null) {
// 把上面获取到的新的windows放置缓存，并返回
sAccessibilityCache.setWindows(windows);
return windows;
}
}
……
}
Starting from iaccessibilityserviceconnection, the IDE starts to prompt cannot resolve symbol 'iaccessibilityserviceconnection', and the tracking cannot be skipped any more. This is because this file belongs to the Aidl file, which is the interface file used for cross process communication in Android. Its specific source code can be seen on Google source. Interested students can take a look: iaccessibilityserviceconnection.aidl. This indicates that the UI automation process has started communication with the accessibilityservice process. We can regard the current program as the client, and the Android system service is the server. From here, we really go deep into the core of the Android system. Below is the library of Android native.
Here, we can summarize with the sequence diagram:
We now know how UI Automator recognizes controls and how to operate control elements? For example, realize the automatic click of the control.
Let's start with the source code. For example, the click action of a control element, in uiobject2 class, the key code is as follows:
public void click() {
mGestureController.performGesture(mGestures.click(getVisibleCenter()));
}
First, the getvisiblecenter method can obtain the central coordinate point of the control node according to the control node information, that is, the accessibilitynodeinfo mentioned above. Then pass the coordinate point to the click method of mgestrure. Here, it is to encapsulate the click action. Finally, it is handed to the performgestrure method of mgestrurecontroller object to implement the click action.
For the click method of mgestrure, this mgestrure is a construction factory. Its click method directly generates a pointergestrure object, which represents the action when performing gesture operation. For example, the start coordinate point, end coordinate point, duration, moving direction, speed and so on.
Let's focus on the performgesture method of the mgesturecontroller object. The key codes are as follows:
public void performGesture(PointerGesture ... gestures) {
…………
// 执行传入的手势操作动作
MotionEvent event; // 这个是关于运动事件
for (……) {
…………
// 初始化运动事件，并调用UI Automation的injectInputEvent注入事件，异步执行
event = getMotionEvent(……);
getDevice().getUiAutomation().injectInputEvent(event,true);
…………
}
…………
}
}
You can see that the event injection is also completed through UI automation. Take a look at the key code of the injectinputevent method,
public boolean injectInputEvent(InputEvent event,boolean sync) {
…………
// 异步执行，这段代码之前有关于锁的操作
return mUiAutomationConnection.injectInputEvent(event,sync);
…………
}
We found that the operation is also performed through a connection. The iuiautomationconnection class corresponding to the connection object also belongs to an Aidl file.
Here is also a sequence diagram,
According to the official description, accessibilityservice means that developers can optimize the use experience of people with disabilities by adding attributes similar to contentdescription without modifying the code. You can open accessibilityservice to try. Click the area and there can be voice or touch prompts to help people with disabilities use the app.
Of course, accessibility service has been broken in China. More and more apps use accessibility service to realize some other functions, even gray products.
In China, the functions realized through accessibility service include free root automatic installation, automatic red envelope grabbing, automatic reply to wechat messages and other black technologies.
Of course, there are also some malicious functions, such as software anti uninstall. When users want to uninstall your app, they usually come to the setting interface, find your app and choose uninstall. If we monitor this page and find that it is their own app, we will exit directly. Won't it be impossible to uninstall? Yes, it's simple, but the malice behind it is chilling.
After reading the above analysis, you may be interested in automation. In fact, the steps are very simple:
It is often said that "interview makes rocket, work screw". In fact, everyone's usual work may be "screwing", but from the perspective of personal career development, it is not desirable. Only by constantly digging their own technical moat can we improve the irreplaceable of individuals. When "screwing in the screws", we might as well look up and see that the whole "rocket" is a unique structure.
https://juejin.cn/post/6844903456809943053
https://developer.android.com/reference/android/app/UiAutomation
https://testerhome.com/topics/1887
https://blog.csdn.net/luoyanglizi/article/details/51980630
https://www.kancloud.cn/digest/uiautomatorpriciple/192698