An Android application for crowdsourcing 3G user experience

An Android application for crowdsourcing 3G user experience Ingenier´ıa en Inform´atica Universitat Polit`ecnica de Val`encia Author: Miquel Mart´ınez Raga Supervisor: Dr. Juan Carlos Ruiz Garc´ıa September, 2011 Abstract This report is composed by a project splited in two parts, a practical part and a research one. The first part of the project was done in Valencia (Spain) under the supervision of the company NUBESIS. We developed an Android application that is the mobile application for a website developed by the same company. We describe the problems we had while developing the application and the way we solved them. We will explain the different processes that take place in the application and how these processes are integrated in the application’s functionality, we also talk about the user’s interaction with the different screens and their behavior. The second part of the project is planned as a research project to improve the connectivity problem that can appear in the first application. This part was done in Sydney (Australia) in cooperation with the University of New South Wales (UNSW) and under the supervision of Professor Mahbub Hassan. In this part we discuss the design and implementation of Android based 3G/HSDPA network bandwidth measurement mobile application. This application acts as a mobile sensor in a crowd sourcing system. We use a network link bandwidth estimation technique called packet pair probing, which can easily be implemented on a mobile platform and we also justify why we have chose the specific methodology after reviewing the related literature. We also propose a measurement initiation process with the Measurement Server which allows the packet pair probing technique to reflect an accurate download bandwidth on the measurement. We have calibrated and fine-tune the measurement tool so it can contribute optimally to the crowd sourcing system by addressing issues such as usability, data consumption and power consumption. We include geo tags in each I measurement we take and discuss the implementation issues addressed in the project. Finally, we introduce an algorithm which measures the download bandwidth in a timely fashion. We study the behaviour of the measurements by changing parameters such as packet size and packet train length. The results obtained were evaluated by comparing them to a reliable commercial bandwidth estimation tool under the same environment. Given these results we conducted a number of hypothesis tests where we used the T-statistic as the test statistic under the null hypothesis. II Acknowledgments It has been a great experience for me to write part of my final project at the Universidad Polit´ecnica de Valencia (UPV) while doing a work placement for the company NUBESIS, and of course it has been amazing to write the other part at the School of Computer Science and Engineering (CSE) at the University of New Sout Wales (UNSW). I would not have been able to complete this project without the support and encouragement of many people. I would like to take this chance to thank all of them. I would like to thank in first place the company NUBESIS for giving me such a great opportunity of working with them, specially to Javier de la Cueva Orts for all the meetings and all the support and comprehension that he offered me since I know him. I think he is a great person and a really hard worker. In second place I would like express my gratitude to Juan Carlos Ruiz Garc´ıa, my supervisor at the UPV. Juan Carlos has been very supportive and he inspired me to go to Australia to do part of my final project. He is an incredible mentor and an extraordinary person, without him none of this would have ever been possible, he is a role model to me. I would like to thank Professor Mahbub Hassan too, he gave me the opportunity to collaborate in his project and also for all the resources he provided me. Mahbub has given me insightful advice, constant support, patient guidance and inspiring ideas on the way to proceed in my project. He is a great man and an even greater mentor. His influence has been key to seeing out the successful completion of this project. And of course, I would like to thank my co-supervisor Dr. Salil S. Kanhere for being so helpful and patient with me. Salil offered me good guidance and support through insightful suggestions and at times contructive critism that opened new avenues to explore and streamlining my research throught. He III has shared with me his knowledge and ideas and for this I am greatful. Salil is a great person and I’m very happy to have had the chance to colaborate with him. I would like to thank also to my colleague Thilanka Panthie who is colaborating with me on this project. Thilanka has been very supportive and patient with me during the duration of this project. His knowledge and willingness to help have proved to be instrumental in the timely completion of this project. Last but not least, I am very thankful to my family for their support and their unconditional love. Without their constant support and encouragement on many a night when the goal did not seem attainable, I would not have been able to complete this project. This report is dedicated to my parents, Miguel Mart´ınez and Elvira Raga, they have been very helpful and they have always beleived in me. Thank you very much. I also would like to thank my girlfriend, Miriam Panero, who has made such tremendous efforts during these months and has given me a lot of strength to keep working in hard times. Miriam, I’m very lucky to have you in my life. IV Contents Abstract I Acknowledgements III 1 Introduction 1 1.1 ReportDescription ........................ 1 1.2 Android .............................. 2 1.2.1 Justifying the system . . . . . . . . . . . . . . . . . . . 2 1.2.2 Androidsystem...................... 3 1.2.3 Android Activity . . . . . . . . . . . . . . . . . . . . . 5 1.3 CloudComputing......................... 6 1.3.1 Description ........................ 6 1.3.2 History........................... 7 1.4 Report organization . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Related work 10 2.1 Introduction............................ 10 2.1.1 Pathrate.......................... 10 2.1.2 CapProbe......................... 11 2.2 Mobile applications . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2.1 Speedtest.net mobile . . . . . . . . . . . . . . . . . . . 12 2.3 What we have learn from the Related Work . . . . . . . . . . 13 3 PideCita Application 14 3.1 Introduction............................ 14 3.2 Certificates and security . . . . . . . . . . . . . . . . . . . . . 14 3.2.1 Definition of the problem . . . . . . . . . . . . . . . . . 14 3.2.2 Preparing the certificate . . . . . . . . . . . . . . . . . 15 3.2.3 Authenticating the request . . . . . . . . . . . . . . . . 17 3.3 Methods and classes . . . . . . . . . . . . . . . . . . . . . . . 20 3.3.1 Classes........................... 20 V 3.3.2 Methods.......................... 22 3.4 GoogleMapsuse ......................... 23 3.4.1 Integrating Google Maps in the application . . . . . . . 23 3.4.2 Interacting with the map . . . . . . . . . . . . . . . . . 24 3.5 Facebook ............................. 29 3.6 Functionality ........................... 30 3.6.1 Introduction........................ 30 3.6.2 Mainscreen........................ 30 3.6.3 List and Map screen . . . . . . . . . . . . . . . . . . . 31 3.6.4 Company File screen . . . . . . . . . . . . . . . . . . . 33 3.6.5 Services screen . . . . . . . . . . . . . . . . . . . . . . 34 3.6.6 Agendas screen . . . . . . . . . . . . . . . . . . . . . . 35 3.6.7 Booking confirmation . . . . . . . . . . . . . . . . . . . 36 3.6.8 Log in process and user screens . . . . . . . . . . . . . 37 4 Connectivity issues 40 4.1 Background of the problem . . . . . . . . . . . . . . . . . . . . 40 4.2 How this problem affects our application . . . . . . . . . . . . 41 4.3 A solution for the problem . . . . . . . . . . . . . . . . . . . . 42 5 Bandwidth Measurement 43 5.1 Introduction............................ 43 5.2 Measurement technique . . . . . . . . . . . . . . . . . . . . . . 43 5.2.1 Serverpart ........................ 43 5.2.2 Clientpart......................... 46 5.3 Conductedtests.......................... 51 5.3.1 Results........................... 51 5.3.2 Hypothesys Tests . . . . . . . . . . . . . . . . . . . . . 53 6 Smartphone application 56 6.1 Introduction............................ 56 6.2 Information collected by the application . . . . . . . . . . . . 56 6.2.1 Measurements . . . . . . . . . . . . . . . . . . . . . . . 56 6.2.2 Measurements file . . . . . . . . . . . . . . . . . . . . . 58 6.3 Device components . . . . . . . . . . . . . . . . . . . . . . . . 60 6.3.1 Location information . . . . . . . . . . . . . . . . . . . 60 6.3.2 Devicedetails....................... 61 6.3.3 Network provider information . . . . . . . . . . . . . . 61 6.3.4 Network connection information . . . . . . . . . . . . . 62 6.4 Issues ............................... 62 6.4.1 GPSaccuracy....................... 63 VI 6.4.2 Data consumption . . . . . . . . . . . . . . . . . . . . 66 6.5 Applicationwork ......................... 69 6.5.1 Interface.......................... 69 6.5.2 Functions ......................... 71 6.5.3 Uploading......................... 73 6.6 Fullprocess ............................ 75 7 Conclusions 78 Bibliography 80 VII Chapter 1 Introduction 1.1 Report Description This report is structured basically in two parts, as mentioned in the Abstract this project has been made in two different places. The first part is the practicall part, while the second is more based on researching to improve the first one. In the first part we talk about the Android application we developed for the company NUBESIS, PideCita. In this part we describe application’s web based features along a full dedicated chapter, we focus on the code and on the visual parts of the application. The most significant feature of the application is that is a cloud computing application, this feature is explained in more detail in chapter 2. The issues found when developing a web based application made us think about the second part. In this second part we design and develop an Android application to collect mobile networks information and upload this information to a server, all the collected information stores the geolocation where the measurements where taken, letting the server stablish patterns and identify areas with lower signals. During the report we will show the relation between both parts. In the next section we justify our decision of choosing Android over other operating systems and we introduce the Android operating system, so the reader won’t get lost with some terminology used in this report. After that we explain how the report is organized chapter by chapter. 1 CHAPTER 1. INTRODUCTION 8 In Chapter 2, we discuss the related work we used as a reference for our work. We discuss the studies based on techniques to estimate the connection bandwidth, which we used to develop our Bandwidth Measurement tool. We introduce a commercial tool used to tests our results and we conclude arguing the ideas we got from the related work. Chapter 3 is related with the Android application we have developed in the first part of the project. In this chapter we talk in detail about the security problems we found when authenticating with the PideCita website. We explain the main methods and classes used in the application, we also discuss the integration of different APIs to our project, Google Maps and Facebook. And we conclude the chapter showing in detail the functionality of the application. Chapter 4 introduce the problem found with the PideCita application that lead us to work on the second part. This chapter introduce the problem we found with the mobile networks and explain how this problem affects the developed application. The chapter conclude with a section that introduce the solution developed and studied during the second part of the project. In Chapter 5, we discuss the measurement tool we developed for the second part of the project. We first give an introduction about the tool and its components. Then we talk about its different components, client and server, focusing on what they do and how they operate. At the end we present some performed tests and we discuss the obtained results. Chapter 6 is related with the Android application we have developed in the second part. In this chapter we explain all the parts that compose the application. We start talking about the way we store the information obtained with the application. We explain the different components and functionalities we need to obtain from the device, and how we get them. We discuss about the issues we had to face while developing the application. We finish talking about the main functions that we developed to achieve our needs, focusing on their individual and group behavior. Chapter 7 is an attempt to complete the jigsaw puzzle by connecting the dots between the techniques used in the second part of the project and the final outcome of the same, and also talking about the benefits of integrating these techniques into the first part of the project. We discuss the main aims of this whole project, briefly outlining why we choose Pair Packet probing over other techniques. We also make reference to our main findings and CHAPTER 1. INTRODUCTION 9 finally conclude by remarking on the future direction of our research. Chapter 2 Related work 2.1 Introduction In this chapter we talk about the related work we make reference to in this report. We discuss studies about techniques to estimate the connection bandwidth which we used to develop our Bandwidth Measurement tool. We also introduce a commercial mobile application that has significant features related to our work. Packet pair probing techniques are introduced as network path capacity estimation methodologies ([3], [4]) using dispersion of two back to back packets [2]. These same techniques can be used to estimate the network narrowest link throughput which is the link between the base station and the network end user in our case. 2.1.1 Pathrate Pathrate was introduced by Dovrolis et.al [3] as a tool which is based on packet pair dispersion techniques. Pathrate uses UDP probe packets to measure the narrowest link capacity. In order to initiate a measurement process it uses a TCP connection as the control channel. Authors have justified their choice of UDP probes as the probing packets instead of TCP based probes such as ICMP and TCP-FIN packets, saying that employing such packets will affect the bandwidth measurements because the reply probes are forwarded on the reverse path from receiver to sender. Successive packet pair probes used in the Pathrate tool have at least RTT difference between them to isolate each packet pair on the way from sender to receiver. Accuracy of Pathrate compared to CapProbe is limited due to its inaccurate readings in high bandwidth paths with narrow links between 640Mbps to 1000Mbps due to dispersion measurement noise at the receiver. Authors 10 CHAPTER 2. RELATED WORK 11 pinpoint that such noises occurs due to application layer time stamping when the bandwidth is much larger than mentioned above. And authors suggest that a machine with higher time resolution (much faster processor) or OS level time stamping could avoid such noises at the receiver by precision time stamps. Also, Pathrate provides inaccurate readings in highly congested or loaded paths because of the large probing packets it employs, it also encounters additional dispersion due to network cross traffic. 2.1.2 Cap Probe CapProbe is a link capacity estimation tool for wired and wireless links. Introduced by Kapoort et.al [4], which is based on packet dispersion techniques as described in section 2.1.1, and combined with an error filtering technique/parameter called minimum delay sum. Minimum delay sum is the the minimum time dispersion between any packet pair out of all the packet pairs and the authors argued that such packet pair is not distorted by cross traffic induced queuing and reflects accurate narrowest link capacity estimates under the assumption that in a packet train there will be at least one such packet pair. Authors have used a modified version of the ping utility (IPUtils open source software) which can send a packet pair instead of one packet. During the capacity calculation stage the program waits for ICMP replies (packet pair) to calculate the narrowest link capacity. Figure 2.1 shows the CapProbe’s packet dispersion technique for calculating narrowest link capacity. It is based on the idea that if two consecutive packets are the same size then the transmission delays are the same for both packets within a link. Figure 2.1: Packet Pair Dispersion Technique Used in CapProbe [4] CHAPTER 2. RELATED WORK 12 Authors argued that a dispersion of a packet pair from source to destination would be compressed or expanded due to cross traffic and hence they provide inaccurate capacity estimations (figure 2.2). The minimum delay sum from at least one packet pair would be enough to measure the capacity of the narrowest link. (a) (b) Figure 2.2: (a) Over estimation and (b) under estimation of capacity [5] 2.2 Mobile applications 2.2.1 Speedtest.net mobile Speedtest.net mobile (ST) is a commercial application developed for Android and iOS devices. This application is the native Android version of a very popular broadband speed test on the internet [6]. It operates using a large global infrastructure to minimize the impact of internet congestion and latency. This application according to the Android Market has been installed by one to five millions users. It is one of Android’s top free applications in the Tools section. Once you begin a test with this application, it tries to get your GPS information via the location service on the device, in case it is unable, it uses the GeoIP information provided by MaxMind.MaxMind provides its geo location technology through the GeoIP brand. They have a GeoIP database which can be accessed from different OS. The tests performed in this application consists in three parts: 1. Latency test CHAPTER 2. RELATED WORK 13 2. Download bandwidth 3. Upload bandwidth We only focus on the “Download bandwidth” part to compare it with our data. While doing a test, the user can visualize a real-time graphs of throughput during the tests. The data collected from the tests can be exported to CV S format and sent by email so users can use the results, which makes it very helpful to compare results. 2.3 What we have learn from the Related Work Below we endeavour to explain why application such as ST or measurement tools like CapProbe and Pathrate can not be used for crowd sourcing. The user should actively participate in order to take measurements from the ST application which consumes valuable time. From our observations a typical ST test will take between eight to twelve seconds depending on the network parameters at the time of the measurement. CapProbe uses ICMP packets and such TCP based measurements would not reflect the same bandwidth measurements suitable for streaming media. On the other hand Pathrate uses UDP packet sizes between 550 bytes and 1500 bytes with 500 to 1000 packet pairs for each individual test which could sink the mobile data plans of contributors demotivating them. For example, ST provides us the service of network bandwidth measurement and users can contribute to the ST’s measurements archive which ST uses for commercial purposes such as sharing bandwidth measurements with network providers and related services. As discussed with regard to Pathrate, in order to timestamp received packets accurately we have to reduce the overhead at the receiver. During the Measurement tool development stage we have identified that our receiver program is burdened with high computational overhead leading to inaccurate timestamps and resulting in large variations of throughput measurements. The packet pair probing technique could underestimate or overestimate due to network path queuing. This may result in higher or lower measurements of the actual rate. Chapter 3 PideCita Application 3.1 Introduction This application is a cloud computing client developed for Android. This application is the mobile version of the cloud computing service offered in this website, http://www.pidecita.com. The application has been developed under the supervision of NUBESIS S.L. (owners and developers of the previous website). This website allows the users to apply for an appointment in any company registered in the system. The companies belong to different areas in the industry, the user’s can choose from medical companies to hairdressers. This website allows users to book for an appointment without the need to contact the company because all companies provide their available timetables, so a user can choose between several companies the one that fits better to his/her schedule. The aim of the application is to make easy and nice the interaction of the users with the system through their mobile phones. In this chapter we explain the different parts of the application and their interaction with the main system. We also talk about the issues we’ve found in the development process and how we solved them. 3.2 Certificates and security 3.2.1 Definition of the problem Due to the fact that this application is based on a cloud computing environment we need to access to the information stored in the server. To obtain the information we need to do GET requests through HTTP to the server 14 CHAPTER 3. PIDECITA APPLICATION 15 https://docs.nubesis.com/bookitit android/. The information is structured in XML files. The main problem resides in the fact that the application needs to accept the server certificate as a trusted certificate. Every request to the server has to contain authentication fields to avoid malicious attacks but Android’s HTTP libraries are the 4.x version of the Apache libraries, meanwhile Java usually uses the 3.x version. The new version of the libraries has some modifications that complicated a little bit the work, we explain all the problems in next sections. 3.2.2 Preparing the certificate The certificate is a document with an electronic signature from an entity allowing anyone to recognize the entity and avoiding impersonations. Keytools allows us to administer our keys and certificates to use them when necessary to authenticate in servers. We can create a “keystore” (KS) where we can store our certificates as trusted certificates, they are considered as trusted because a public key is attached to this cert and it can be verified anytime needed. ... # # List of providers and their preference orders (see above): # security.provider.1=sun.security.provider.Sun security.provider.2=sun.security.rsa.SunRsaSign security.provider.3=com.sun.net.ssl.internal.ssl.Provider security.provider.4=com.sun.crypto.provider.SunJCE security.provider.5=sun.security.jgss.SunProvider security.provider.6=com.sun.security.sasl.Provider security.provider.7=org.jcp.xml.dsig.internal.dom.XMLDSigRI security.provider.8=sun.security.smartcardio.SunPCSC security.provider.9=sun.security.mscapi.SunMSCAPI security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider # ... Code 3.1: List of providers When adding our certificate to our KS Java uses JKS (Java Keystore) by default and that is a problem when using the KS in Android. Android uses by default BC (Bouncy Castle) as its KS provider, so the easiest way to solve CHAPTER 3. PIDECITA APPLICATION 16 this problem was to use the same KS provider when storing our certificate in our KS when using the Keytools tool. The only thing needed is to download the .jar file with the required libraries and modify the “java.security” file that can be found in our Java folder (in Windows XP: Program Files\Java\jre\lib\security\) and add the provider to the providers list, it would look like figure 3.1. After that we have to navigate to the Java folder where the keytool program is (this is only necessary in Windows) and execute in the terminal the following code: > keytool -importcert -v -trustcacerts -file \path_to_cert\certificate.crt" -alias \Alias for the cert" -keystore "keystore_path\myKeystore.bks" -provider org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath "path_to_jar/bcprov-jdk16-145.jar" -storetype BKS -storepass mysecret Code 3.2: Storing the certificate If the KS does not exist the program create a new one in the specified path “keystore path\myKeystore.bks”. The -storepass refers to the password that is going to be used for the KS and that we will need to access it. The instructions in 3.3 let us check the information contained in our KS. keytool -list -keystore "keystore_path\myKeystore.bks" -provider org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath "path_to_jar/bcprov-jdk16-145.jar" -storetype BKS -storepass mysecret Code 3.3: Storing the certificate Once the KS is created we import it to our Android project, the right path to import it is inside our raw folder, in this case would be res\raw\docsnubesis.bks. The extension of the KS (bks) makes reference to the provider used in the creation. CHAPTER 3. PIDECITA APPLICATION 17 Figure 3.1: Keystore in raw folder Now that the KS has been created and is in our project, we have to access it to create a HTTPS socket that we will associate to our HTTP client. ... DefaultHttpClient httpclient = new DefaultHttpClient(); KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType()); InputStream in = this.getResources().openRawResource(R.raw.docsnubesis); try { trustStore.load(in, "nubesis".toCharArray()); } finally { in.close(); } SSLSocketFactory socketFactory = new SSLSocketFactory(trustStore); Scheme sch = new Scheme("https", socketFactory, 443); httpclient.getConnectionManager().getSchemeRegistry().register(sch); ... Code 3.4: Use of the certificate to create the socket 3.2.3 Authenticating the request When doing a request to the server we need to authenticate the request by attaching the authentication code of the request in a header. The message authentication code (MAC) is obtained by using Hash-based Message Authentication Code (HMAC), which is an specific construction to calculate CHAPTER 3. PIDECITA APPLICATION 24 The last thing we have to do is to obtain the Android Maps API key for our application, otherwise the MapView will not work properly. A single Maps API key is valid for all applications signed by a single certificate, so in the debugging state we use the Android debug certificate, this certificate is found in the SDK folder. The way to obtain the Maps API key is by registering the MD5 fingerprint of the certificate in this website http://code.google.com/android/maps-api-signup.html. To obtain the fingerprint we need to use keytool and the debug.keystore that we will find in the Android folder, Code 3.9 shows how it needs to be done. $ keytool -list -keystore ~/.android/debug.keystore ... Certificate fingerprint (MD5): 94:1E:43:49:87:73:BB:E6:A6:88:D7:20:F1:8E:B5:98 Code 3.9: Obtaining the certificate fingerprint After introduce the fingerprint and accept the terms and conditions we get our Maps API key for that certificate and we have to copy it in the field “android:apiKey” in out MapView object in the layout. If we change the certificate then the map will not work, that means that when publishing an application in the market, a private suitable key needs to be obtained and we can no longer use the debug key when publishing the application, so we need to obtain a new Maps API key in that situation. 3.4.2 Interacting with the map Once the map is integrated into our application we need to use it, between the information received about a company we also get a latitude and a longitude. In this section we explain how we managed to place in the map all the companies that a user gets when he performs a search, we first talk about the graphical part and show how it looks, and then we focus on the code. The purpose of the map in the application is to let the users inspect the real position of the companies in relation to their position. When a user looks for a group of companies in a concrete field in the industry the application shows a list of those that are inside the specified distance ratio. The method that performs this action is “companiestag”, explained in section 3.3.2. Figure 3.2 shows the resultant list in a search, this is the result of a request prepared for testing, so some companies are fake and their coordinates are random. Those companies with the yellow star are the ones CHAPTER 3. PIDECITA APPLICATION 25 Figure 3.2: List of companies chosen by the user as favorite companies, for this reason there are two icons to distinguish between companies on the map screen, one to represent the favorite companies and the other one for the rest. If the user is not logged in then all the companies appear as not favorite. (a) Company (b) Favorite company Figure 3.3: Company icons Those companies that are considered as favorite by the user are displayed on the map with the icon 3.3b and the others with icon 3.3a. On the map screen when the user taps over an icon a bubble with the name of the company appears, a second tap on the bubble takes the user to the description screen of the company, if the second tap is in an area where there is no company then the bubble hides. In figures 3.4a and 3.4b we can see the representation of the companies on the list over the map. As mentioned before, these companies (their locations and data) are made up. CHAPTER 3. PIDECITA APPLICATION 26 (a) Zoom in (b) Zoom out Figure 3.4: Companies displayed on the map To interact with the MapView we need to use a class named Overlay. The Overlay class is the class that place the items on the map. We have three classes involving the maps use: CMapChoices, MyOverlay and MyLocation. CMapChoices is the main Activity that displays the map and the top bar with the four buttons that let us choose if we want to display only those companies that has offers, those which are favorites for the user, or both of them. The other two buttons are one to the companies list and the other to change the distance ratio for the companies to be shown. The CMapChoices class extends the Activity class and it is the only one that can access to the resources of the project. To control the map we have an instance of MapView and another one for MyOverlay (mapview and overlay respectively). There is also a List of MyLocation, mapLocations, that allow us to obtain from a company only its name, location and know if it is a favorite company. The class CMapChoices is the one that receives the companies list when it is called, so we defined a function called getMyLocations() that iterates the array that contains all the companies and creates for each one a MyLocation object with the required information to fill mapLocations. In code 3.10 we can see how this function works. CHAPTER 3. PIDECITA APPLICATION 27 public List<MyLocation> getMyLocations() { /** * We fill the array of locations from the information of * the companies we have in the array @someCompanies, where * we store the given companies from our previous screen. **/ if (mapLocations == null) { mapLocations = new ArrayList<MyLocation>(); for (int i = 0; i < someCompanies.size(); i++) { mapLocations.add(new MyLocation(someCompanies.get(i) .get_sName(), someCompanies.get(i).get_dLatitude(), someCompanies.get(i).get_dLongitude(), someFavoriteNames .contains(someCompanies.get(i).get_sName()))); } } return mapLocations; } Code 3.10: Function getMyLocations When the CMapChoices Activity is onCreate the last thing we do is prepare the variable overlay to place all the companies on the map. To do this only two functions need to be used, one to create a new object of the class MyOverlay and the other to add the overlay to the map. •overlay = new MyOverlay(this) •mapview.getOverlays().add(overlay) All the process to place the items on the map is defined in the MyOverlay class. When we create a new MyOverlay object we pass as parameter the reference to the CMapChoices Activity, doing so we can access the project’s resources like images and views, that means we can have a reference to the MapView, because we will need to modify it and place the items over it. When the instruction “mapview.getOverlays().add(overlay)” is executed in CMapChoices then the function draw is called for the overlay variable. We override this function and define two new functions that will be called from this one: drawMyLocations and drawInfoWindow. The first one is the function that place all the companies on the map and the second one is the responsible of drawing the pop up bubble with the name of the company when the user taps over the icon. Next we explain the code of the function CHAPTER 3. PIDECITA APPLICATION 28 drawMyLocations because its significance for the map, in code 3.11 we can see the code. private void drawMyLocations(Canvas canvas, MapView mapView, boolean shadow) { /** We draw the locations in the map, we also consider if * the location belongs to a favorite company or not to * choose the image we want to draw. **/ Iterator<MyLocation> iterator = mapLocationViewer.getMyLocations().iterator(); Point screenCoords = new Point(); while (iterator.hasNext()) { MyLocation location = iterator.next(); mapView.getProjection().toPixels( location.getPoint(), screenCoords); if (location.isFav()) { canvas.drawBitmap(favoriteIcon, screenCoords.x - favoriteIcon.getWidth() / 2, screenCoords.y - favoriteIcon.getHeight(), null); } else { canvas.drawBitmap(bubbleIcon, screenCoords.x - bubbleIcon.getWidth() / 2, screenCoords.y - bubbleIcon.getHeight(), null); } } } Code 3.11: Function drawMyLocations What we are doing is creating an iterator for the list of companies we want to place on the screen, each company is stored in a MyLocation object that makes reference to the company and its coordinates are stored as a GeoPoint, which is a class that represents a pair of latitude and longitude. We need to convert this coordinates to onscreen pixel coordinates, for that we use the function “mapView.getProjection().toPixels(location.getPoint(), screenCoords)”, it needs two parameters, a GeoPoint, which contains the coordinates we want to convert and a pre-existing object of the class Point for the output, where the onscreen pixel coordinates will be stored. The object screenCoords contains the coordinates we need, the problem is that if we try to place an image using that point, that point would be the reference for the top left corner of the image, so we need to modify the coordinates to place the image as we want. The if else condition is to check whether to draw the CHAPTER 3. PIDECITA APPLICATION 29 favorite or the regular icon for the company. When the user interacts with the top buttons (Distance, Favorites and Offers) the set of companies can change, maybe we need to display only those with an offer, or only the favorite ones, or simply the set changes because the distance ratio has changed and there are new companies on the list. In all this situations what we do is to modify the main ArrayList that contains all the companies and call the function “redraw()”, this function is the responsible of redrawing the canvas, so all the new companies will replace the previous ones. 3.5 Facebook In the web version of the application, the user can log in using a Facebook account, so there is no need to register on the website and have a specific user and password for the website http://www.pidecita.com. The application pretends to offer the same facilities to the user, so we integrated a way to log in through Facebook. To use Facebook in our application we need an application id for Facebook, because the application is the same as the website both have the same application id. We let the user choose between log in with Facebook or with PideCita from the Login screen (Figure 3.5). When the user logs in with Facebook then we can access to the user’s personal information necessary to validate the user against the user’s database, we use the method validatefacebookuser(Table 3.1) to obtain the PideCita user’s id linked to this account, if the user does not exist then a new user is created and the user’s id is returned. When the user log in correctly on Facebook then we proceed to obtain the information by doing requests using the Facebook API. The methods used are called synchronously and they will block the UI, so instead of that we use a class provided by the Facebook SDK called AsyncFacebookRunner, which perform asynchronous calls that do not block the UI. The method returns a JSON object that we need to parse in order to obtain the Facebook id of the user and the name, we will also need the phone number when using validatefacebookuser, but due to the fact that not everybody have a phone number in their Facebook account, what we do is obtaining the device’s phone number if any, if the phone number can’t be obtained then we set a default number. The default number is useful when creating a new PideCita user linked to CHAPTER 3. PIDECITA APPLICATION 30 this Facebook account, but later on the user will need to change it because it is necessary that the users provide a real phone number in case the companies need to contact them. Figure 3.5: List of companies 3.6 Functionality 3.6.1 Introduction In this section we describe the transitions between the application’s screens. There are several differences between the available options a user can use depending whether the user is logged in or not, because of that we only consider when a user is already logged in to explain the screen transitions but pointing out when necessary what a not logged in user can not do. We explain the process of applying for an appointment from beginning to end going through all the possible screens. After this we explain the authentication process for a user and the accessible screens related to the user’s account. The data used for the application’s screen captures showed during next sections is fake data prepared for test purposes. 3.6.2 Main screen In this screen we have the icons (tags from now on) of the type of companies the user can search for. This screen contains four default tags: Sports, CHAPTER 3. PIDECITA APPLICATION 31 Cosmetic, Health and Add. The first three make reference to a type of companies, with the last one the user can add more tags to the main screen, by doing this the user can do faster a search for a type of companies. In figure 3.6 we can see the process the user has to do to add the tag “Layers” to the main screen. Figure 3.6: Add a tag to the main screen These new tags added to the main screen can be deleted easily, the user only needs to press and hold over the tag and a pop up will appear with one question and two answers to choose, Q:“Do you want to delete the tag?”, A: “Yes” and “No”. When a tag is added to the screen then it is removed from the list of available tags, and when it is removed from the screen then it is added back to the list. When a tag is pressed the application executes a request to the server to obtain the list of companies that match with the type indicated by the tag. This request is processed and the information is shown in a new screen called “Listchoices”. 3.6.3 List and Map screen The user is able to check the resultant companies either in a list form or in a map. Both screens have a top buttons bar with for buttons, three of them do not change between screens (Distance, Discounts and Favorites) but the other one does, the left button is used to change between these two screens. Here we explain the functionality of these buttons: CHAPTER 3. PIDECITA APPLICATION 32 Figure 3.7: Performing a search Distance This button is used to change the distance limit from our position. When this distance changes a new request is performed with the new distance. Discounts Its function is to show only those companies that has an offer. Favorites When pressed it shows only those companies among the list that are favorites for the user. This button is not enabled when the user is not logged in. In figure 3.7 is visible the kind of information the user can see from a company in the list: name, address, distance from user, if it has a discount, if it is a favorite company and the company picture. The last one is represented in the test data as a green round ball with a white confirmation mark inside. If the user is not logged in then all the stars appear in grey. For the next example we used a small group of companies that contains all the different possibilities, it is easier to see the difference on the map when a button is pressed with a reduced group. Figure 3.8 and figure 3.9 show the different transitions in each screen when the user press the buttons Discount or Favorites. A user can see the detailed information of a company by clicking on it, CHAPTER 3. PIDECITA APPLICATION 33 Figure 3.8: List screen transitions Figure 3.9: Map screen transitions at the list screen the user only need to tap over the company row but in the map the user need to tap the icon of a company, after that the way to access to the information is by taping over the pop up that shows up right after the first tap. All the information is displayed in a new screen called “Ficha T´ecnica” (Company File). 3.6.4 Company File screen This screen shows the information related to the selected company and also about the offer they have in case there is any. The user can check the details and decide whether to choose or not this company. A logged user can add Chapter 4 Connectivity issues 4.1 Background of the problem During this last decade, the area of mobile phone devices has experienced a significant evolution. Mobile Networks (MN) have improved by leaps and bounds. The way current devices make use of MN has opened the doors to a new generation of interaction between users and their phones. This progress in MN has made the number of mobile devices increase very rapidly in the past few years. That lead us to a situation where there is a large number of users using their devices to access the Internet. MN operators have tried to adapt to the constant evolution of mobile technologies in order to guarantee a higher customer satisfaction, sometimes this is not possible and users experience connection problems, which causes dissatisfaction. The lack of consistency in MN is due to environmental and technological limitations such as load variations, the number of cell towers, your surrounding environment etc. We are aware that these problems are more significant in some areas than in others. So users can not expect a good and consistent network coverage all the time. Inconsistent network behavior greatly affects users’ Quality of Service (QoS), especially when they use applications that require a high use of data such as multimedia streaming services. Network coverage is not always consistent due to the problems mentioned before. The number of cell towers in an area can vary significantly from one area to another. For example. mobile phone providers do not have equal coverage in the same country. This is influenced by how young a mobile provider is in a particular country and by the population in that area. Also, 40 CHAPTER 4. CONNECTIVITY ISSUES 41 some companies do not invest money in areas with a small population. For example, in Australia, the main cities have signal for every provider, however this is not the case in some small towns. The situation will likely continue unless companies decide to expand their presence uniformly throughout a region. So, considering an area with signal of a certain provider, when moving through that area, the signal reception can vary due to the problems we afore-mentioned. These problems affect the MN, particularly web based applications through users’ QoS. This is due to the fact that applications can’t know in advance these changes, so they can’t prevent the consequences related to them. 4.2 How this problem affects our application The Android application developed for NUBESIS, PideCita, is an application with cloud computing features, and for that reason one essential feature on the phone that must work properly is the Internet connection. In our application all the information is stored in a server and accessed from the device through HTTP requests, that makes things more comfortable for the user because all the information is available from anywhere and the risk of loosing something does not exist. The users can check all the appointments, modify their favorite companies or apply for new appointments either from the phone or from the computer. When the users use their applications on the phone they expect them to be fluent and solid, if the it is not a web based application that depends only about the phone features, then the fluency between screens or the fact that something works wrong may depend only on the phone’s hardware, and there’s nothing we can do about that. But when it comes to applications with a continuous use of the Internet, then a bad connection can affect to the QoS the user expects from the application. In our case all the information displayed on every screen is obtained through requests to the server, and the time this request needs to obtain the information can be the difference between a fluent screens transitions or a slow one. If we talk about the PideCita application, these problems can cause unexpected behavior on the application, for example, if the user wants to apply for an appointment and after several minutes thinking about which company to choose, the day and the time, if the connection is down when the user wants to confirm the date then this will not happen. In this kind of situations the users do not know if the problem is due to the application, the CHAPTER 4. CONNECTIVITY ISSUES 42 server or just because the connection is down at the moment, the only thing the user cares about is about the time lost on applying for an appointment and the upraising frustration from not achieving the result. That kind of things can make users do not want to use the application, and this is not good for business. 4.3 A solution for the problem We know our problem depend on the cell phones coverage, and that’s something we can not change. What we can do is try to study them and prevent this lack of coverage in some areas, and this take us to the second part of the project. Applications can’t find out by themselves the quality of the device’s connection in advance, so our idea is to build a server that will contain information related to the MN signal for different providers in a certain location. This allows applications like multimedia streaming to take advantage of this information by preventing a lack of signal in the area the user is moving towards, thus they can adjust their data flow in order to minimize a loss in the QoS. All the information obtained is stored in a database at the server and applications will access this information through HTTP requests. In order to collect this data, we have developed an Android application. The advantages of doing so is that we can use crowd sourcing to collect the data. By spreading the application, we can get a large number of users contributing to the database by collecting information about the MN signal in their respective areas. This application makes use of the GPS and several phone features that help us to collect all the information about the MN in a certain location. We built a server that we use to estimate the available bandwidth of the phone. During next chapters we focus on this application, how it works and explain its behavior. Chapter 5 Bandwidth Measurement 5.1 Introduction One of the main aims of this project is to be able to measure the download bandwidth (DBdw) in a certain location. Android doesn’t provide any class to do so for a 3G connection, so we have developed a server in Java that supplies the client with packet trains of data which we will use to measure the download bandwidth that the user can get. In this next section we discuss how this server/client works and we talk about the techniques used to measure the bandwidth and the tests we performed to interpret our results. A thing to consider when measuring the DBdw, is that the user is attached to a data plan. So one of our main consideration was the data consumption when trying to measure the DBdw because we want the users to use the application with minimal affect to much their data plan. 5.2 Measurement technique 5.2.1 Server part The server will have to supply requests from different users, so we developed a multi thread server. The behavior of the server is shown in figure 5.1. In the server we have a main thread that is always running and listening for new requests. In this main thread we open a DatagramSocket that listens through port “5555”. The DatagramSocket class is for using UDP socket connections. Even though TCP is a more reliable protocol, when trying to establish a pure TCP connection with the Socket class from Java and 43 CHAPTER 5. BANDWIDTH MEASUREMENT 44 using 3G in the device, we saw that the connection was never establish in 3G although it was in Wifi. This is caused by the providers’ NAT (Network Address Translation). Many mobile carriers use NAT and related technologies, so there is no guaranteed way to make a direct socket connection between two devices or a device and a PC. Figure 5.1: Multi thread server As mentioned earlier, the server is requested to send a packet train of a certain size to the client and each packet with a specific size. The values for the possible number of packets in a train are 10, 20, 50, 100 and 200. The packet size is expressed in bytes and the possible values are 50, 100, 300, 500 and 1400. These values are stored in two different arrays, npckts for the number of packets in a train and size for the different packet sizes. The requests from client to server are send in UDP packets. When a new request arrives, we extract the data from the packet. The data contained in the request consist in two numbers, these numbers are indexes for the previous arrays. The first index is the position of the desired packet size and the second index refers to the train size. To serve the clients requests we have implemented a runnable class called Handler. To create an instance of this class we need to pass it some parameters. The way this is done from the main thread is shown next. CHAPTER 5. BANDWIDTH MEASUREMENT 45 DatagramSocket udpsocket = null; try { udpsocket = new DatagramSocket(obtainport()); } catch (Exception e) { e.printStackTrace(); System.out.println("Problems creating socket"); } new Thread(new Handler(udpsocket,npckts[p],size[s], packet.getAddress(),data.substring(0, size[s]))).run(); Code 5.1: Handling clients requests The parameters are described as follows: DatagramSocket socket The socket created to send packets through a different port number than the main socket. int packets The number of packets that we need to send (train size). int size The size of each packet in bytes. InetAddress address The IP address of the client. This address is obtained from the packet received in the request. String data The data of the packets we are sending. In the client this data is not treated so this string is full of blanks. The number of blanks is the same than the number of bytes the packet’s data must have. As we can see in Code 5.1, we first create the socket in a port given by the function obtainport(), which gives an unused port between “3000” and “3300”. The number of packets and the packet size is obtained from the arrays mentioned before, npckts[p] and size[s]. In sand pare stored the numbers obtained from the client’s request packet. From that packet we obtain the client’s IP address with packet.getAddress(). The data for sending the packets is obtained from the variable data. This is a string created with a number of blanks equal to the maximum packet size possible, in our case this is 1400 bytes, hence it contains 1400 blanks. We use the “substring” function to pass a string that contains only the required size. The function run() in the Handler class is responsible to perform the function of sending the packet train to the client. We only create one packet and we send it as many times as the train size. We don’t need to create new packets because as mentioned before and as we explain in next section, CHAPTER 5. BANDWIDTH MEASUREMENT 46 the client does not need to treat the data from the packet. The process for sending the packets is as follows. public void run() { try { byte[] bufer = new byte[size]; bufer = data.getBytes(); DatagramPacket packet2 = new DatagramPacket(bufer, bufer.length, address, 5555); for (int i = 0; i < packets; i++) { udpsocket.send(packet2); } } catch (Exception e) { System.err.println("Error sending the train"); }finally{ udpsocket.close(); udpsocket.disconnect(); } } Code 5.2: run() function in Handler class 5.2.2 Client part The client is developed in Java and is developed for running on an Android system. As we did with the server, we are using Java network classes to define client’s behavior. This method is implemented in a class called BandwidthMeasure and it implements the Runnable interface. Now we explain how it works and what it does. When the DBdw is calculated, the client creates a thread passing a new object of the BandwidthMeasure class which is created with three parameters. The first parameter refers to the number of packets we want to receive in a train of packets (train size), the second parameter refers to the size of each packet in the packet train and the last parameter is a reference to the application’s main Activity that called the thread. As we mentioned before, the first two parameters are the values that we attach to the packet we send in our request, the server then services our request by sending a packet train with the train size we want with the requested size for every packet. The third parameter is used in order to access public functions defined in the main Activity. This particular function is called addbdw(float bdw), its purpose is CHAPTER 5. BANDWIDTH MEASUREMENT 47 to receive the estimated DBdw in the thread and store it in a global variable called estimation, this last variable will contain the estimated DBdw. This functions will be explained in next chapter. We use the same arrays that the server uses (size and npckts) in order to know the size of the packet and the number of packets we are expecting. When the function run() is called, the first thing we do is initialize all the positions of received times array to “-1” and all the positions of ports array to “0”. The received times array is of type “long” (because the arriving time is stored in nanoseconds, so we need long numbers) and it’s used to register the arriving time of each packet, so with this information we calculate the delays between packets. The ports array is of type “int” and it’s used to store the number of the sending port of each received packet. Their size is the same as the number of packets we are expecting to receive. Thereafter, we create and send the request to the server then we wait for the response. In Code 5.3 we can see the procedure to wait for the response. We first set a timeout value for the socket of “time” milliseconds (this value is by default setted to 1 second). The reception of the first packet may take a long time due to the traffic, that is why the timeout for the first packet is higher than for the rest. After the first packet arrives, we change the timeout value to half a second, that value is established considering the delay between packets we would experience in a situation with a very low bandwidth. We initialize the three variables that we will use as conditions for the while loop. In goodpackets we count the packets we have received, later, we will need it inorder to calculate the bandwidth as can be seen in Code 5.4. The variable init is used to store the initial time of the request and timeout to record if the timeout occured. The conditions for the while loop to iterate are: 1. The number of received packets is lower than the expected. We stop looping when we achieve as many packets as we expected. This value is stored in npckts and numofpac is the value of the parameter we received in the object’s constructor and it refers to the train size. 2. When the timeout occurs, we set the timeout variable to true and we stop looping. 3. Knowing the initial time we can make sure that we are not receiving packets for longer than the specified time. So if the difference between the last packet received and the initial time is higher than ten seconds (109nanoseconds) we stop receiving. CHAPTER 5. BANDWIDTH MEASUREMENT 48 udpsocket.setSoTimeout(time); init = System.nanoTime(); goodpackets = 0; boolean timeout = false; try { udpsocket.receive(pac); received_times[goodpackets] = System.nanoTime(); ports[goodpackets] = pac.getPort(); goodpackets++; udpsocket.setSoTimeout(500); } catch (InterruptedIOException e) { timeout = true; } while (goodpackets < npckts[numofpac] - 1 && !timeout && (received_times[goodpackets - 1] - init) < 10000000000f) { try { udpsocket.receive(pac); received_times[goodpackets] = System.nanoTime(); ports[goodpackets] = pac.getPort(); goodpackets++; } catch (InterruptedIOException e) { timeout = true; } } Code 5.3: Receiving a packet train What we do is for every packet that arrives to our system we register its arriving time in received times and the sender’s port number in ports. Every arriving time and port number for each packet are stored in the position of their respective arrays which matches the arriving order of the packets, considering the first packet number “0”. When we finish waiting for more packets, we close and disconnect the socket. Inorder to calculate the bandwidth we check if certain conditions are fulfilled. If the DBdw is low, the time to receive one hundred packets is very high, so if we stopped receiving because the ten second condition occured, then we make sure that all packets belong to the same train and then we calculate the DBdw. If this is not the case and the number of packets is higher than eighty (eighty packets give similar values to one hundred), then we calculate the bandwidth. In both situations when we calculate the DBdw we pass it to CHAPTER 5. BANDWIDTH MEASUREMENT 49 the main Activity through the function mentioned before. This call uses the object mainActivity, which is the reference to the main Activity received as a parameter when the BandwidthMeasure object was created. This call is as follows: mainActivity.addbdw(calculateBDW(received times)) The parameter received refers to the array mentioned before, received times, the parameter ports refers to the array ports and the parameter packets makes reference to the variable goodpackets, which contains the number of packets received. Sizeinbits contains the size in bits of one single packet, we add eight more bytes because it’s a UDP packet and these eight bytes correspond to its header. In the first main “if” we check whether the first and the last ports are the same. If they are different, we try to find the index of the first packet with the same port number as the last packet. We add one to first because we start calculating the delay between a packet and its previous one. The second main “if” is used to make sure that if we ended receiving and the ten second condition is not true, then if we have at least 80 packets we then estimate the DBdw. If the ten second condition is true, we then consider less packets to measure the DBdw because for lower DBdw the estimation can be obtained accurately with less packets. If any of the main “if” conditions is true, then we iterate over the arriving times calculating the delays between every two packets from begining to end and we accumulate these delays in the variable delays. We check if delays is “0” or not inorder to return the estimated DBdw. In dlyinsecs we store the accumulated delay in seconds and then we use it to return the estimated DBdw in bits per second (bps). Chapter 6 Smartphone application 6.1 Introduction In this chapter we describe the application we developed. The contents of this chapter will go as follows. First, we introduce the different elements we measure with the application and describe the file we upload to the server. Afterwards, we describe the different components and information about the device that we are accessing and their role in the application. Further, we discuss the main issues we had and how we solved them. In the end we explain how the application works and its interaction with the users. 6.2 Information collected by the application In this section we describe the information we are obtaining from the application and how we upload it into the database server. We first briefly look at the different data we store for each measurement one at a time. Afterwards, we discuss in more detail the structure of the file that contains all the measurements and the uploading process. 6.2.1 Measurements The application we developed retrieves information related to the device. This information can be divided into four different groups: device, location, network provider and network connection. Now we identify and define them. In later sections we talk about how we obtained them and their role in the application. We must mention that in addition to this data we also save the date, starting time and ending time for each measurement but explanations are left out for simplicity’s sake. 56 CHAPTER 6. SMARTPHONE APPLICATION 57 1. Device Device identifier This number is the IMEI for GSM and the MEID or ESN for CDMA phones and is unique for each device. Phone type Name of the radio type the device uses. 2. Location Latitude & Longitude These are the coordinates of the device’s GPS in the moment a measurement is taken. Accuracy Represents the accuracy in meters of the coordinates received from the GPS. 3. Network provider Operator identifier Known as a ”MCC / MNC tuple”, this number is a combination of two numbers, the Mobile Country Code (MCC) and the Mobile Network Code (MNC). The MCC is the ISO country code of the operator and the MNC makes reference to the operator in that country (This is explained in more detail in section 6.3.3). 4. Network connection Local area code A unique number assigned to a “location area”, where this “location area” is a set of base stations that are grouped together to optimise signaling. Base station identifier Identifier of a base station in a “location area”. Signal strength This number represents the signal strength and its value is between 0-31 or is equal to 99 as defined in TS 27.007 8.5 (3GPP Technical Specification). Network type Depending on whether the phone is a GSM or a CDMA phone, this value lets us know what kind of network is in use (GPRS, EDGE, UMTS, HSDPA, . . . ). Bandwidth Represents the download bandwidth when the measure took place and it is obtained with the tool mentioned in chapter 5. Instead of uploading every single measurement that a user makes, what we do is register each measurement in a Java class we created called Measurement. This class contains a field for all information required in a measurement and CHAPTER 6. SMARTPHONE APPLICATION 58 it provides setters and getters to access these values. In the main class we have declared an ArrayList of Measurement objects (measurementset), so all the measurements are stored in this array till the file is upload to the server or till the user exits the application, in that case we write the measurements into the file. Using an array makes it very easy to add/delete new measurements and access them. Once the file is uploaded, the array is cleared to avoid uploading the same measurements twice. 6.2.2 Measurements file All the measurements taken while the application was running are saved in an XML file called “UNSWBandwidthData.xml”. This file is stored in the external memory of the device, in a folder created by the application. So if the application is removed, then the folder and all its contents are also removed. The structure of the file and how it is treated by the application is explained below. The file’s structure is easy to understand now that we know the values it contains. Here we describe the relationship between the tags and the measurement values. In figure 6.1 we can see an example of a file that contains two single measurements. List of tags: <measurements>Start tag of the document, its attribute “user phone id” is the device identifier of the phone that uploads the file. <measure>First tag for each measurement and has two attributes: “date” that represent the date of the measurement in “dd/mm/yyyy” format and “time”, which represents the starting time of the measurement in “HH:mm:ss” format. <endtime>The time when the DBdw estimation finished in the format “HH:mm:ss”, which is the ending time of the measurement. <BaseStationId>The value contained in this tag is the base station identifier. <LAC>Local area code of the base station. <operatorid>The MCC+MNC of the network operator. <signalstrength>The value of the signal strength when that measurement was taken. CHAPTER 6. SMARTPHONE APPLICATION 59 <phonetype>The type of phone, GSM or CDMA. <networktype>The kind of network connection we have (GPRS, EDGE, UMTS, HSDPA,. . . ). <lat>The latitude of the coordinate given by the GPS. <long>The longitude of the coordinate given by the GPS. <accuracy>The accuracy in meters of the geographical coordinate. <bdw>The value obtained in kilobits per second from the Measurement tool (Chapter 5). When the application is started, it checks if the file exists. If so, that means it contains measurements from previous runnings, so adding new measurements would produce a malformed .xml file. We extract the information about the previous measurements by parsing the file and we store them in the ArrayList<Measurement>we mentioned in previous section (6.2.1). With all these measurements stored, the application is ready to start obtaining new data that will be added to the previous array. Figure 6.1: UNSWBandwidthData.xml CHAPTER 6. SMARTPHONE APPLICATION 60 When we stop the application, we must save all the measurements we took. What we do is iterate over the ArrayList and write them in the file as shown in figure 6.1. When we write the file, it does not matter if the file exists or not because we are not appending information to it. At the time the user decides to upload the file, if the upload is successful we delete it to avoid future conflicts with the data. We explain in section 6.5.3 the uploading process. 6.3 Device components The application requires access to several pieces of information provided by the device: location information, device details, network provider information and network connection information. In this section, we describe them one by one and we refer to the Java classes used in each case. 6.3.1 Location information One important thing for us is to know the user’s location, so we can see the relationship of the obtained data in a certain location. Hence, in the later phase of the work we will be able to build a map with this data. When developing a location-aware application for Android, developers can use GPS and Android’s Network Location Provider to acquire the user location. Although GPS is more accurate, it only works outdoors and it consumes more battery power than others. We are using GPS because it fits better to our purposes, we also need to be as accurate as possible and GPS gives more accurate results. The Java class we are using is LocationManager, this class provides access to the system location services. These services allow applications to obtain periodic updates of the device’s geographical location. This updates are obtained by means of callback. We define a LocationListener that we pass to our LocationManager and this listener must implement several callback methods. The LocationManager calls these methods when the user location changes or when the status of the service changes. We get the updated location through the method onLocationChanged(Location location) and we get the new location in the Location parameter. The behavior of our application when the location is changed is defined in this method. CHAPTER 6. SMARTPHONE APPLICATION 61 In section 6.4.1 we discuss in more detail the updating time for the GPS listener and its relationship with the accuracy of the obtained coordinates. 6.3.2 Device details A way to identify the user’s contribution to our database is establishing a relationship with a user and its phone. The best way to do that and at the same time minimize the user’s interaction with the application is by using the device identifier, which is unique for every device. Storing the relation between the user’s email and user’s device identifier in the database, we can know who uploaded the data to the server and how much this user contributed to our database. The access to information about the telephony services on the device is provided by the T elephonyManager (TM) class. Once this class is instantiated to the telephony service of the device, we can easily get this identifier by calling the method getDeviceId(). This value is part of every measure we upload to the database and it is a primary key of the data in the database. 6.3.3 Network provider information We can’t expect that all users use the same network provider, so due to the fact that there are different network providers, we need to know which one the user is using when doing the measurements about the network. This will allow us in later work to classify the obtained data in different sets and compare the data we collected between the different providers. To access this information we use the same class we mentioned before, T M. We get this information by calling the method getNetworkOperator() and we get the numeric name of the current registered operator. This number is a combination of two numbers, the Mobile Country Code (MCC) and the Mobile Network Code (MNC), also known as a ”MCC / MNC tuple”. The MCC is the ISO country code of the operator and is part of the International Mobile Subscriber Identity (IMSI) number, which uniquely identifies a particular subscriber and is stored on a (usually) removable SIM card. It is always three numbers and in our case it’s 505, which is the Australia’s MCC. The MNC is always used in combination with a MCC and uniquely identify a mobile phone operator in the country referenced by the MCC. In our tests we were using a Vodafone AU SIM card and the value of the tuple is “50503”. CHAPTER 6. SMARTPHONE APPLICATION 62 6.3.4 Network connection information We need to collect information related to the phone’s network for our database. This information is obtained from the network features, such as the signal strength, the phone type (GSM or CDMA), the network type (CDMA, UMTS, EDGE, . . . ) and also information related to the current base station, the base station identifier and local area code (LAC). The LAC is a unique number assigned to a “location area”, where this “location area” is a set of base stations that are grouped together to optimize signaling. To get this information we use the T M class to obtain the first features (signal strength, phone type and network type) and we are using another class called GsmCellLocation for the other features. Phone type and network type can be obtained by doing a simple call to getP honeT ype() and getNetworkType(), respectively, from our T M object. Signal strength on the other hand, requires overriding a callback function in PhoneStateListener. This listener must be registered in our T M object and it is used for monitoring changes in specific telephony states on the device. So, when registered in our TM object, we pass the listener and a flag indicating which state we want to monitor (in this case, LISTEN SIGNAL STRENGTHS). Overriding the onSignalStrengthsChanged(SignalStrength signalStrength) method lets us control the behavior when the signal strength changes. To get the information about the base station another callback function needs to be overridden and another listener flag is required (this time it is LISTEN CELL LOCATION). With this flag we supervise the changes in the device’s cell location and by overriding onCellLocationChanged(CellLocation location) we get the related information to the cell location (stored in location). Inorder to obtain the information about our current cell location, we need to cast the CellLocation object into a GsmCellLocation object, so then we can call two functions (getCid(),getLac()) to obtain that information. 6.4 Issues While developing the application we found some issues regarding the best way to get our measurements. Here we identify them and explain how we managed to solve them. These issues are related to the GPS accuracy and with the amount of data consumed from the user’s mobile plan. CHAPTER 6. SMARTPHONE APPLICATION 63 6.4.1 GPS accuracy The accuracy of the coordinates obtained from the GPS are affected by the amount of time the GPS signal is alive. The GPS needs a start up time but this time is not specified anywhere because it may vary from one GPS to another. So we performed a test to find out how to get a high level of accuracy without affecting the battery consumption greatly. The test took place at the UNSW and it consisted of running an application to obtain the coordinates give by the GPS and its accuracy. We ran this application four times and each time with a different updating time for the location. Our values where 5, 10, 30 and 60 seconds. The test was run for 14 minutes for 5 and 10 second intervals and for 28 minutes for 30 and 60 second intervals. The length of the data set is different for each one, this is due to the fact that when the updating time is shorter we get more GPS coordinates per minute. The graphs shows the results through the time. First, in figure 6.2 we show the frequency of the results for each updating time. As can be seen, when the GPS updates its location every 5 seconds, more than 95% of the coordinates obtained are at least 20 meters accurate and more than 35% are at least 10 meters accurate. For the other updating times, between 60% and 70% of their results are 20 meters accurate. If we have a look to figure 6.3a, we can see the variations in the accuracy for 30 and 60 second updating times. Even though we get some accurate results for 60 seconds it is quite irregular and the same happens for 30 seconds. This is caused by the behavior of the GPS, it is designed to save battery, so when the time interval is that long, the GPS gets a location according to its updating time and then its status is set to “STOP”. When the updating time passes, it changes its status to “START” because of these changes, the accuracy of the location obtained is affected and is not always as accurate as it could be with a shorter time. By looking at figure 6.3b, we see that for 10 seconds there are still large variations but for 5 seconds the values obtained appear to be more regular. CHAPTER 6. SMARTPHONE APPLICATION 64 Figure 6.2: Accuracy frequency (a) 30s vs 60s (b) 5s vs 10s Figure 6.3: Accuracy comparison To show the results in figure 6.4 we used the data obtained during the first 14 minutes for 30 and 60 second intervals inorder to compare them with the results obtained from 5 and 10 second intervals. CHAPTER 6. SMARTPHONE APPLICATION 65 Figure 6.4: Cross data We conducted another test with a time interval of 500 milliseconds, as we expected the results where more accurate than the rest but this time interval has a much higher consumption of battery. In figure 6.5 we can see the difference with 5 seconds. Figure 6.5: 5 seconds vs 500 milliseconds CHAPTER 6. SMARTPHONE APPLICATION 72 as explained in section 6.4.2. When storing the Network type, we first check if the network we are connected is mobile or wifi, if it’s wifi then we set its value to “WIFI”. onLocationChanged(Location location) This function was explained in section 6.3.1, but now we’ll see how it works. Figure 6.10: onLocationChanged(Location location) The variables current and last are objects from the Calendar class. This variables store the time, so in time we store the difference between the cur- CHAPTER 6. SMARTPHONE APPLICATION 73 rent time and the last time we did a measurement. If this time is higher than 15 seconds and the network connection is available, we proceed to do a measurement. Then we check the global Bandwidthtask object (band), if it is still null (it is the first time we try to run it) or it not running, then we do a measurement. The process to do a measurement can be seen in figure 6.10. First we create a new Bandwidthtask object and we execute it. Meanwhile it is running in the background we store the information related to the GPS (accuracy, latitude and longitude) and we create a new Measurement object, which we add to measurementset and we store the reference to that object in measurement, which is a global variable of the class Measurement. Then we pass this Measurement object to the get3Ginfo(Measurementmeasure) function to obtain the rest of the required information. 6.5.3 Uploading For uploading the file we created a class called UploadingT ask that extends AsyncTask (explained in section 6.4.2). In this class we override two methods: doInBackground and onPostExecute. Now we explain the code used for uploading the file to the server (Code 6.1). try { HttpClient httpClient = new DefaultHttpClient(); HttpPost request = new HttpPost( new URI("http://129.94.172.130/")); MultipartEntity entity = new MultipartEntity(); entity.addPart("upfile", new FileBody(Measurements_file)); request.setEntity(entity); HttpResponse response = httpClient.execute(request); int status = response.getStatusLine().getStatusCode(); if (status == HttpStatus.SC_OK) { Measurements_file.delete(); return 1;// "Upload Complete"; } else { return 2;// "Couldn’t upload, pleas try later..."; } } catch (Exception e) { return 3;// "There was a problem while trying to upload"; } Code 6.1: doInBackground() What we do is create an HttpClient and prepare the POST request with the CHAPTER 6. SMARTPHONE APPLICATION 74 server’s IP address. We can’t simply attach the file to our request with the standard Java libraries for http connections, it is a little bit tricky. Instead, we are using two third party libraries called “apache-mime4j-0.4.jar” and “httpmime-4.0-beta1.jar” that implemented methods that already embrace all the work. These libraries allow us to use the MultipartEntity class and with this class we can attach the file to our request by setting the request’s entity to it. After uploading, if the response is “OK” (200), then we delete the file. if (checkNetwork()) { if (!uploadtask.getStatus().equals(Status.RUNNING)) { if (!uploadtask.isCancelled()) uploadtask.cancel(true); if (Measurements_file.exists()) { Toast.makeText(this, "Wait while we upload the file", Toast.LENGTH_LONG).show(); uploadtask = new UploadingTask(); uploadtask.execute(this); } else Toast.makeText(this, "There’s no data to Update", Toast.LENGTH_SHORT).show(); } } else Toast.makeText(this, "You need connection to upload the file", Toast.LENGTH_SHORT).show(); Code 6.2: Upload File Code 6.2 shows the behavior when the “Upload File” button is pressed. When the user press the button we try to upload the file. Before we upload the file we check that all the conditions for uploading are satisfied. These conditions refer to the network connection and to the UploadingTask object. We check if the network connection is active and then before launching a new UploadingT ask object we make sure there’s not any other object already running. So, if the file exists, we create a new task and we execute it to upload the file. We use a global object from this class, uploadtask, by doing so we can check its status and make sure we don’t start two tasks at the same time. If the file does not exists or network connection is not available, we print a message to give some feedback to the user. CHAPTER 6. SMARTPHONE APPLICATION 75 6.6 Full process To explain how the application works we first introduce a transition diagram with the different states of the application. Then we show how the measuring part works with a flow chart and we explain its behavior. In the transition diagram the flow is shown from the application launch. After we initialize the variables we check if the file UNSW BandwidthData.xml exists. If so, we can upload or start measuring (“Start or Upload” state), if it does not exists then we can only start the measuring part (“Start” state). When we press the “Start” button, it is possible that there’s no connection or the GPS is disabled, in that situation we reach a state which we will only leave when both of them are enabled. When everything is enabled and the “Start” button is pressed, then we reach the “Measuring” state where we collect the data. We stop measuring when the user presses the “Stop” button. When we stop measuring, if the measurementset array is empty (so there are no measurements to store in the file), we return to the “Start” state, otherwise we go to the “Start or Upload” state. When we are in the “Start or Upload” state and we try to upload the file, if the upload is successful we move to the “Start” state, if the upload is unsuccessful then we stay in this state. This behavior is presented in figure 6.11. Figure 6.11: States Diagram for the application The behavior of the “Measuring” state is shown in figure 6.12. When the “Start” button is pressed we check the network connection and if it is active, CHAPTER 6. SMARTPHONE APPLICATION 76 then we register the LocationListener with a 5 second updating time. If it is not active then we move to the “No network or No GPS” state and afterwards we go back to the calling state. Setting the updating time to 5 seconds means that every 5 seconds the GPS updates its location and this location is received in onLocationChanged(Location location). The behavior of this function is explained in section 6.5.2. When the “Stop” button is pressed, we then check the array where we store all the measurements (measurementset), if it is not empty, then we store the measurements in the xml file UNSWBandwidthData.xml and we go to “Start or Upload” state. If the array is empty, then we go to the “Start” state. While measuring, if the GPS is disabled, we check if any measurements have been taken, if so we store them in the xml file and then we move to the “No network or No GPS” state. If there are no measurements then we move straight to this state. CHAPTER 6. SMARTPHONE APPLICATION 77 Figure 6.12: Measuring flow Chapter 7 Conclusions Finally, we attempt to complete the jigsaw puzzle. We have noted the meteoric rise of mobile smart phones. As a consequence, applications for these smart phones, particularly those related to multimedia streaming are in huge demand. This necessitates the need for better and more consistent data throughput to provide a better user experience. This report is about how we have designed and implemented a mobile application which collects user’s perceivable download data bandwidth by acting as a crowd sourcing sensor on behalf of a geo-intellegent system, and also about how developed a web based application that would perfectly be able to use these features for itself. The application developed in second place measures 3G/HSDPA download data bandwidth actively and automatically thereby reducing user interaction and data consumption. By integrating this functionality in the PideCita application will lead us to more measurement contributors, eventually resulting in a vast measurement database. Efficient mobile phone power consumption and the user data consumption were our two main considerations. This presented an optimization problem. On the one hand we wanted to maximize GPS accuracy but at the same time, we also wanted to minimize the power consumption. We therefore endeavoured to find an optimal balance between GPS accuracy and power consumption whilst minimizing the consumption from the user’s current data plan. Adding the measurement functionality to PideCita can make things easier, as we are using an HTTPS connection for our requests we could use the same request to measure the bandwidth, avoiding like this an extra connection. Packet Pair probing is the base technique used in our measurement ap78 CHAPTER 7. CONCLUSIONS 79 plication. Packet Pair probing was a convenient choice because of its low computational overhead. Also, it allows for the ability to reconfigure the measurement tool at anytime with very little effort and the fact that it is capable of providing highly accurate results only serves to reinforce this point. When running our tests for the measurement tool we used measurements obtained from the commercial mobile application Speedtest.net mobile as a benchmark. We tested the hypothesis that the results obtained from our application are equal to the results obtained from Speed Test using a two-sided T-test. We conclude that our measurements lie within the 95% confidence interval around the mean of the results obtained using Speed Test, 80% of the time. In the future we hope to be able to implement the PideCita application on other mobile platforms. This will not only extend our reach in the market, merging both applications will also facilitate access to a wider range of contributors which will serve to enrich the quality and increase the size of the database. Concluding, adding these new features to the PideCita application would imply that the application would be more prepared to handle connection problems. Users won’t suffer the connection problems as many times as they do now, this would benefit this application and also other web based applications. Other future applications developed by NUBESIS could use this information and become more reliable. Bibliography [1] Android Developers, http: // developer. android. com/ index. html . [2] Constantinos Dovrolis, David Moore, and Parameswaran Ramanathan, What do packet dispersion techniques measure? [3] Constantinos Dovrolis, Parameswaran Ramanathan, and David Moore, Packet dispersion techniques and a capacity estimation methodology. [4] Rohit Kapoor, Ling jyh Chen, Alok Nandan, Mario Gerla, and M. Y. Sanadidi, Capprobe: A simple and accurate capacity estimation technique for wired and wireless environments, UCLA Computer Science Departament, Los Angeles, CA 90095, USA. [5] Rohit Kapoor, Ling jyh Chen, M. Y. Sanadidi, and Mario Gerla, Accuracy of link capacity estimates using passive and active approaches with capprobe, UCLA Computer Science Departament, Los Angeles, CA 90095, USA. [6] SpeedTest.net, http: // speedtest. net/ . 80