Fast CPU’s with multi-tasking OS platforms, high sensitive GPS receivers, 3G / 4G wireless communication chipsets, high-resolution digital video cameras, touch screen LCD displays and large storage size are common in smartphones. The use of MEMS sensors is then no longer limited to existing applications such as screen rotation, power saving, motion detection, E-Compass and 3D gaming. More advanced applications for MEMS sensors, such as Augmented Reality (AR), Location-Based Services (LBS), Pedestrian Dead-reckoning (PDR) are currently being developed.
This article discusses the role of MEMS sensors in advanced applications in mobile devices including Mobile Augmented Reality (MAR), LBS and the solution of MEMS sensor fusion integrated with a GPS receiver to determine the position and orientation using the dead-reckoning method.
Augmented Reality
Augmented reality (AR) is not a new topic. By definition, AR is a feature or user interface that is implemented by the interaction of superimposed graphics, audio and other sensing enhancements over a real-world environment that is displayed in real-time, making it interactive and manipulable. The Integration of 3D virtual information into real-world environment helps provide users with a tangible feeling of the virtual objects that exist around them.
Recently, there have been a few successful applications of AR. For example, vehicle safety application can provide information on road conditions and surrounding cars by projecting these to the windshield. Another application is to display information of an object such as restaurant or supermarket, etc., when the smartphone is pointed to the object with known position and orientation. Also, one can find the nearest subway station in a big city by moving the phone fully round 360 degrees, locating the subway and following the directions to the destination.
Social networking is playing a key role in peoples’ current life. When approaching a shopping center, a user can point the phone to the shopping center sending friends virtual information augmented on his location and the surrounding environment. Vice versa, the user will have information on his friends’ whereabouts. Therefore, AR is a new way of changing the feeling to the real world.
The key components available in smartphones for MAR are shown in Figure 1.
Figure 1. Smartphone structure for MAR
* Digital video camera: Used to stream information about the real-world environment and display captured video on the LCD touch screen. Currently 5-Megapixel or higher camera sensors are available in new smartphones.
* CPU, Mobile OS, UI and SDK: Components are the core of a smartphone. 1GHz or higher dual-core CPU with 512MB RAM and 32GB storage space can be found in new smartphones. UI and SDK give developers a simple way to call APIs to access the graphics, wireless communications, database and MEMS sensors raw data without knowing the details behind during their own applications development.
* High-sensitivity GPS Receiver, or A-GPS or DGPS: Fixes the user current location in terms of latitude and longitude when significant satellites are captured. A lot of effort has been invested over the years to increase the GPS sensitivity and positioning accuracy for indoor and urban canyon areas when satellite signals are degraded and multipath errors occur.
* Wireless link for data transmission including GSM/GPRS, Wi-Fi, Bluetooth and RFID: Provides Internet access to retrieve the database online of the object that is near-by the current location and to give a rough information about positioning while waiting for GPS fix or if GPS is not available. Other short-range wireless links such as WLAN, Bluetooth and RFID can also be used for indoor positioning with adequate accuracy if the transmitters are pre-installed.
* Local or online Database: Utilized for virtual object information augmented on the real-world video display. When the object is aligned to the current position and orientation, its information can be retrieved from online or locally saved database. Users can then click the hyperlink or the icon on the touch screen to receive more detailed information.
* LCD Touch Screen with digital Map: Provides high-resolution UI that displays real-world video augmented by virtual object information. With the digital map, users can know the current location with street names and don’t need to wear special goggles.
* MEMS sensors (ACC, MAG, GYRO and PS): Self-contained components that work anywhere and anytime. Due to low cost, small size, lightweight, low power consumption and high performance, these have become popular for pedestrian dead-reckoning (PDR) application to obtain indoor and outdoor position and orientation with the integration of GPS receiver. The following sections will discuss their key roles in how to increase the accuracy of indoor position and orientation.
The main challenge of the MAR is to obtain accurate and reliable position and orientation anytime and anywhere to align the virtual objects with the real world.
Indoor position and orientation detection
Although many smartphones have a built-in GPS receiver that works well for outdoor location and driving direction displayed on a digital map, many times GPS receivers cannot get a position fix indoors or in urban canyon areas. Even in outdoor environments, GPS cannot give accurate orientation or heading information when a pedestrian or car is not moving. Also, GPS aren’t able to distinguish small height changes. And, moreover, GPS cannot provide the mobile user or vehicle attitude information such as pitch/roll/heading with a single antenna.
Differential GPS (DGPS) is able to obtain a few centimeters of position accuracy, though it needs a second GPS as a base station to transmit in a certain range coarse/acquisition code as the reference position to the mobile GPS receivers. Assisted GPS (A-GPS) can help to some extent for the GPS to get a fix indoor, but sometimes A-GPS still can’t provide an accurate position in an acceptable time interval. With at least three GPS antennas, it’s possible for the GPS to detect attitude information when the mobile user is not moving. However, there is very little feasibility to have multiple GPS antennas in a smartphone.
As a result, a GPS-only smartphone is not capable of providing accurate position and attitude for a mobile user. Self-contained MEMS sensors are an excellent option to assist the GPS for integrated navigation systems for indoor and outdoor LBS.
Modern GPS receivers have an absolute position accuracy of 3 to 20 meters when the antenna has a clear view of sky. It doesn’t drift over time. Strapdown inertial navigation system (SINS) based on MEMS sensors can provide accurate position in a short time, but it will quickly drift over time depending on the performance of the motion sensors. PDR is a relative navigation system based on step length and orientation to calculate the distance traveled for indoor navigation from initial known position. The position accuracy doesn’t drift over time, but the heading accuracy needs to be maintained in a magnetic-disturbed environment and the step length needs to be calibrated by the GPS for acceptable location accuracy.
Based on SINS theory, inertial sensors (3-axis ACC and 3-axis GYRO) are categorized as navigation-grade, tactical-grade and commercial-grade according to their stability of the inherent biases and scale factors. The horizontal position error from unaided ACC only and GYRO only can be calculated from the following two equations [1].
The above equations can be used to calculate the typical inertial sensors performance and the corresponding horizontal position error from their long-term bias stability characteristics. These errors will not grow over time when integrated with GPS. Other error sources such as misalignment, non-linearity and temperature effect, which will cause extra position errors, should also be considered in these calculations.
Recent advances in MEMS processes, MEMS ACC and GYRO have been continuously providing higher performance and nearer to the level of tactical-grade devices. In a short time period such as 1 minute, unaided ACC and GYRO can give relatively accurate position measurements. This is useful to form GPS/SINS integrated navigation systems when the GPS signal is blocked.
Usually for consumer electronics five percent of error on distance travelled is acceptable for indoor PDR. For example, when the pedestrian walks 100 meters, the error should be within 5 meters. This requires the heading error to be within ±2° to ±5° [2]. For instance, when heading error is 2°, then the position error for 100 meters traveled distance will be 3.5 meters [= 2*100m*sin (2°/2)].
In addition, MEMS pressure sensor is able to measure absolute air pressure with respect to sea level. Therefore, the altitude of mobile user from 600 meters below sea level to 9000 meters above sea level can be determined to aid GPS height measurement [2]. Figure 3 shows the PDR block diagram for MEMS sensors and GPS.
Figure 2. PDR block diagram in a mobile device
MEMS sensor fusion
Sensor fusion is a set of digital filtering algorithms to compensate the disadvantages of each individual sensor and then output accurate dynamic attitude information pitch/roll/heading. The purpose of sensor fusion is to take each sensor measurement data as input and then apply digital filtering algorithms to compensate each other and output accurate and responsive dynamic attitude results. Therefore, the heading or orientation is immune to environmental magnetic disturbance as to the bias drift of the gyroscope.
Tilt compensated E-Compass, which consists of a 3-axis ACC and a 3-axis MAG, can provide heading with respect to earth magnetic north. But this heading is sensitive to environmental magnetic disturbance. With the installation of a 3-axis GYRO, it is possible to develop 9-axis sensor fusion solution to maintain accurate heading anywhere and anytime.
When designing a system using ACC, GYRO, MAG and PS, it is important to understand the advantages and disadvantages of each MEMS sensor as shown in the table below.
* ACC: It can be used for tilt compensated digital compass in static or slow motion and it can be used for pedometer step counter and to detect if the system is in motion or at rest. However, an ACC cannot differentiate the true linear acceleration from earth gravity components when the system is at motion in 3D space and it is sensitive to shake and vibration.
* GYRO: It can continuously provide rotation matrix from system body coordinates to local earth horizontal coordinates and it can aid the digital compass for heading calculations when the MAG gets disturbed. But the bias drift over time leads to unlimited attitude and position error.
* MAG: It can calculate absolute heading with respect to earth magnetic north and can be used to calibrate the gyroscope sensitivity but it is sensitive to environmental magnetic interference fields.
* PS: It can be used to tell which floor you are on for indoor navigation and aid GPS for altitude calculation and positioning accuracy when GPS signal is degraded but it is sensitive to wind flow and weather conditions.
Due to the above considerations, the Kalman filter appears today as the most common mathematical instrument to fuse the information coming from the different sensors. It weights the different sensors contribution most heavily where they have the best performances, thus providing more accurate and stable estimates than a system based on any one medium alone [3].
Currently, quaternion based extended Kalman filter (EKF) is a popular scheme for sensor fusion because quaternion has only 4 elements compared to rotation matrix with 9 elements and it can also avoid the singularity issue that is present in the rotation matrix [3].
Conclusion
The main challenge for advanced mobile applications, such as the AR, is accurate position and orientation anywhere and anytime because the AR is closely related to the PDR or the LBS. With the limitation of GPS receiver, MEMS sensors are an attractive solution for indoor PDR since most of these sensors are already available in smart phones.
In order to achieve the allowable five percent indoor PDR position error, MEMS sensor fusion algorithms need to be developed to compensate the disadvantages of each sensor. As the performance of MEMS sensors is continuously improving, the user-independent SINS/GPS integrated navigation system will be common in smart phones in the near future.
Information is shared by www.irvs.info
