Weekly Report 9

This was the final week before our demo. We have tested the application over and over again and removed bugs. My duty was to test the Android part. We, Çiğdem and I, found some bugs in Gallery part and fixed them. We prepared some scenarios too for both image processing part and Google Maps part.

Weekly Report 9

This week, I obtained the relevant images from the database and passed to the image processing part by means of various mysql queries. After compeleting the project me and Guliz tested the product and cleaned the bugs. We prepared scenarios for the final demo and filled the database with images ,their locations and features of them. We are glad to say that we finalized our project, enjoyed a lot while developing it and learned many issues about groupwork, image processing and Android and those were invaluable experiences.

Weekly Report 8

This week I prepared the features of the image processing part so as to insert them to the database. In this way, we were not going to process each image in the database when we do the similarity check using the picture received from the server. That was a challenging process, because I was not able to find an example of saving opencv features into the database on internet, documentaations or manuals of openCV. There were two critical structures called cvSeq and cvMat and unlike cvSeq, the cvMat, which corresponds to a matrix, will easily be saved as an xml document. But cvSeq was collecting objects where cvMat was a union of float double and other basic types. Therefore, converting one into another was not easy to figure out. Finally I decided to divide cvSeq's properties into different cvMat objects and then saving those cvMat matrices into different xml files. Also recovered those xmls to cvSeq's by first converting them to different cvMat matrices and recovering old cvSeq objects by merging those various cvMat matrices. I prepared two different programs, one for preparing xml files and other for recovering cvSeq objects. And I integrated the xml to cvSeq part into the image processing module in order to speed up the application.

Weekly Report 7

This week I tried to integrate the image processing module to the project, after testing the surf method in-depth. Since image processing should be as fast as possible, I implemented it in c++ using opencv library. But I implemented various parts of the server side in java for the sake of simplicity. Therefore, I had two options during integration process:

1. using java native api
2. calling the c++ program as a process

First I tried to use native api. After being successful on a simple hello world program I tried to link dll's of opencv and compile the c++ code with native api. But there were so much dependencies on various dll's and it was hard for me to manage integration with native api. Then I decided to use a java process. The resulting code is as follows:

Weekly Report 8

This week I worked on implementing the camera functionality of our product. Simulating the camera was a bit hard, but in the end I was able to do it in a proper way successfully.

First we have to make things clear. Android does not use the web cam to simulate the camera but when you correctly implemented it, you will get a default image from the platform and this will prove us that we are on the right way. First we have to give permission to the application to use camera functionality. We do it by adding “” that line to the android manifest.

Then we will get a camera from the system.
mCamera = Camera.open();

We will set the parameters of the camera:
Camera.Parameters params = mCamera.getParameters();
mSize = params.getPictureSize();
params.setPictureFormat(PixelFormat.JPEG);
mCamera.setParameters(params);

Then, we have to implement the callback function of the camera:
Camera.PictureCallback photoCallback = new Camera.PictureCallback() {
public void onPictureTaken(byte[] data, Camera camera) {
writeTextOnPicture(data);
}
};

With the above function we will get the default picture as a byte array. We will convert it to Bitmap object then will set the view to that image and we are done.

Weekly Report 7

This week I worked on sending MMS feature of our product. Actually, sending MMS is easy to implement.

Uri mmsUri = Uri.parse("file://"+"/sdcard" + IMAGE_FOLDER + this.pictureName + EXTENTION);
Intent intent = new Intent(Intent.ACTION_SEND);
intent.putExtra("sms_body", "Hi how are you");
intent.putExtra(Intent.EXTRA_STREAM, mmsUri);
intent.setType("image/png");
startActivity(intent);

We will add the picture we want to send as MMS and for that we need the content URI of the file. After that we will add a text to the MMS. However, Android emulator does not support sending MMS option. Normally, we were to type the port number of another emulator inside the “to” part of our emulator and the message would be sent immediately to the other emulator. I tested it and saw that it really doesn't work. However I was able to send SMS message to the other emulator when I deleted the image from the MMS message. This proved that the code I implemented is correct.

Weekly Report 6

This week, I worked on extracting the name of a place from Google Maps by using the GPS information. For that we have to first extract the longitude and latitude of the location in a proper way.

public void extractGPS() {
LocationManager lm = (LocationManager) getSystemService(Context.LOCATION_SERVICE);

Location location = lm
.getLastKnownLocation(LocationManager.GPS_PROVIDER);
if (location != null) {
latitude = location.getLatitude();
longitude = location.getLongitude();
}

lm.requestLocationUpdates(LocationManager.GPS_PROVIDER, 1000L, 500.0f,
this);
}
With that code, we will get the GPS information from the emulator. After that we can get the name of the landmark. We start with giving permission to our application to use the Google Maps API by adding those lines to the android manifest:





First we get a geocoder object.
Geocoder geocoder = new Geocoder(this, Locale.getDefault());

Then, we will extract the address for our current location by using the geocode object.
List

addresses = geocoder.getFromLocation(latitude,longitude, 1);

We will get the first line of the address, the rest will be discarded.
Address address = addresses.get(0);

Finally, we will extract the most meaningful part of that adress.
address.getFeatureName();

That's it. That way, we will get the name of the landmark.

Weekly Report 5

In week 5, my duty was to enhance the client-server communication. Last semester, we were restricted to send only small images to the server and this was a critical obstacle for our project since our aim is to do image processing on the images sent from client. Image processing on small images would restrict our project, so we focused on finishing the enhancement first.

The methods we tried together with Çiğdem were not successful at all. However this week, we discovered that the solution is to add the image as an entity to the HttpPost object. After that longitude and latitude values will be added as a header to this object and execute method will be run.

Weekly Report 6

As Zephyr team, we worked on two topics this week. While Tolga and Huseyin were trying to implement MediaWiki module of the project, Güliz and I were coping with GoogleMaps. At the beginning, Güliz and I tried to implement the GoogleMaps Reverse Geocoding in the server side. But because of the limitations of GoogleMaps API (like the need of a domain name) we couldn't achieve to implement that module in the server side and we had to develop it at client side. Also the advantage of implementing GoogleMaps Api was avoiding unnecessary traffic between client and server.

Geocoder is the class for handling geocoding and reverse geocoding in android. Geocoding is the process of transforming a street address or other description of a location into a (latitude, longitude) coordinate. Reverse geocoding is the process of transforming a (latitude, longitude) coordinate into a (partial) address.

We mainly used:

List adress

getFromLocation(double latitude, double longitude, int maxResults)

Returns an array of Addresses that are known to describe the area immediately surrounding the given latitude and longitude.

and

the addres class which represents an Address, i.e, a set of Strings describing a location.
and the functions we used were:

String getFeatureName()

Returns the feature name of the address, for example, "Golden Gate Bridge", or null if it is unknown 

After we implemented google maps module, we integrated it into the system which looks into the system for the the picture's future name. Now, the system first uses GoogleMaps to get future name of a point if it is unsuccessfull, it searches the database.

Hüseyin and Tolga also completed their tasks.

Weekly Report 5

In this week I searched a segmentation algorithm for the image processing part of the project. Due to huge domain space of the picture, which we will use, one segmentaiton algorithm is not enough for our aim. A fusion approach should be used and the candidate segmentation algorithms for this approach are region growing methods, edge detection based methods, histogram based methods and clustering based methods. Combining these algorithms gives us a better segmentation. After the completion of segmentation part, feature extraction is the next step. For each segmented image a set of features are extracted. By using extracted features a classifier should be designed. Simple classifiers are enough for our purpose such as Bayesian decision making method or decision tree.

Weekly Report 4

This week we tried to carry our project one step farther. Last week we distributed our workload using trac and started to perform our duties. Together with Güliz, we enhanced the client-server communication of our project. The problem in the server client part was caused by the buffer and string size limitations. We partly solved the problem considering those limitations. Now we can send larger images, but it was not sufficient enough. Apart from that, I took part in compass group and looked for a method to send a rotation event for compass of emulator to capture. But documentations for send event command were not satisfactory. But still, we managed to connect to android terminal by using the command line:

telnet localhost 5554

then after we connect to android terminal we sent the swap event:

send event EV_SW:0:0

and

send event EV_SW:0:1

to make a 90 degrees rotation and a rotation in reverse direction. But we could not be able to handle any event using android sensor listener.

Lastly, I helped Tolga during his server-side work which consists of image information retrieving from image database according to the GPS location.

Finally, for this week's demo we achieved sending meeningful information (name of the place) from server to client using gps information.

Weekly Report 4

I worked on the compass information retrieving from Android operating system. Android operating system provide this system as an event listener process. The difficulty is here to simulate the android telephone event in the emulator. The emulator has no ability to provide compass events to the process except the terminal event flopping commands. Therefore, we have used a tool, named SensorSimulator, to create the events needed by our compass information extraction. In addition to that the communication problem still continues. Due to the problem in the android application, large size data cannot be transfered from the client to the server. JSONObject solutions are applied to that problem. However, we don't come up with a solution yet. The experiments continue. A solution is not far away

Database Connection - Weekly Report

Last week, we distributed the responsibilities of each member using Trac. This week, my duty was to establish the database connection.

Firstly, I compiled some data about some famous landmarks including Anıtkabir, the Sheraton Hotel and the Eiffel Tower. In accordance with the entity relationship diagram we presented in the detailed design report, I populated some tables in the database which had already been created before. Now, the Location table holds GPS and compass information, the Imageinfo table contains information about the images and the images themselves, and Has_location table maps the locations with image infos. The SendPicture servlet is now capable of receiving latitude and longitude info of the image from the client and after querying the database using these values, responding to the client successfully with the image description found.

Tolga AKIN
Department of Computer Engineering
Middle East Technical University

Weekly Report 4

This week I worked on several tasks including enhancement of the client-server communication, obtaining compass information and sending GPS location.

For enhancing the communication between the client and the server, I can say that Cigdem and I are close to finding a solution to the file size problem. We enhanced the communication a bit, that is we are now able to send bigger images, but this is still not enough. In the code that we presented in the first demo, we had used String and StringBuffer objects to keep the image, now we realized that these objects have size limitation which prevents us from keeping the image properly.


In order to obtain compass data, we should be able to simulate the behavior of a magnetic compass. That's why we have to find a way to change the direction of the emulator. An application named SensorSimulator simulates this behavior. We have to first install an apk file to the android emulator, then run the Java application of the simulator. After that, we run the apk and connect the emulator and the running Java application. Now we are able to change the sensor values of the emulator. However, when we close the android application, that connection vanishes making it impossible for us to update the compass sensor value.


I also worked on obtaining the GPS information and sending it to the server. Now, depending on the GPS information, server will return a meaningful response to the android client.

Weekly Report 3

We have worked on the configuration management plan report in this week. In addition, I have researched about the image processing part. The k-means clustering algorithm is commonly used in computer vision as a form of image segmentation. The results of the segmentation are used to aid border detection and object recognition. We can use this algorithm to detect object for feature extraction. However, the standard Euclidean distance is insufficient in forming the clusters in our case due to space dimension histogram. Therefore, Mahalanobis distance can be used instead of Euclidean distance to form clusters better in large histogram space dimensions. ISODATA algorithm is a good modified version of k-mean clustering algorithm. It uses both an iterative version of native k-mean clustering algorithm to find the best number of clusters (K) and Mahalanobis distance.

Reference:

Fatos T. Yarman-Vural, Ergin Ataman: Noise, histogram and cluster validity for Gaussian-mixtured data. Pattern Recognition 20(4): 385-401 (1987)

Configuration Management Report - Weekly Report 3

This week we were dealing with the configuration management plan report. When we build computer software, change happens. And because it happens, you need to control it effectively. So, with configuration management we tried to provide a set of activities that are designed to control change by identifying the work products that are likely to change, establishing relationships among them, defining mechanisms for managing different versions of these work products, controlling changes that are imposed, and auditing and reporting on the changes that are made.

Weekly Report 3

This week, all members of our group worked on the Configuration Management Plan Report.
The purpose of this Configuration Management (CM) Plan is to provide an overview of the organization, activities, overall tasks, and objectives of Configuration Management for our application. It addresses configuration item (CI) identification, change control and configuration audits at a high level; additional details regarding CM activities, techniques, and tools should be provided in the CM-related procedures.

We specified the tools we are using, and explained the workload further.

Weekly Report 2

Last week, I worked on the problem of sending large image files to the client from Android part. We haven't come up with a solution yet, but still working on it.

GoogleMaps Api problem and alternatives

There is a problem with the authorization system of Google Maps API. It requires a domain name to activate the product. In the web site it is said that

"For most applications we recommend that you simply register your domain name. Your key will be valid for that domain, its subdomains, all URLs on hosts in those domains, and all ports on those hosts.

For example, if you sign up for a key using http://www.mygooglemapssite.com/, your key is valid for:

• http://www.mygooglemapssite.com/
• http://www.mygooglemapssite.com/mysite/

However this key will not be valid for:

• http://mygooglemapssite.com/
• http://host1.mygooglemapssite.com/
• http://host2.mygooglemapssite.com/mysite

If as recommended you request a key for http://mygooglemapssite.com/ your key will be valid for all of the URLs listed above.

Note that a key for http://www.mygooglemapssite.com/ will only be accepted when the site is accessed using this address. It will not be accepted if the site is accessed by IP address (eg. http://10.1.2.3/) or by a hostname that is aliased to www.mygooglemapssite.com using a DNS CNAME record"

Therefore, we need to search other alternatives to Google map or find a solution to that authorization problem. JXMapViewer can be an alternative to Google Maps API. The JXMapViewer is an open source (LGPL) Swing component created by the developers at SwingLab. It is used for displaying maps on swing panels. On the other hand, Google Earth application already uses Google Maps API as a native PC program, so there should be a way to use it without a domain. This fact encourages us to search more about this problem.

First Demo Contents

This demo consists of extracting the most similar image to a reference image from between a set of subject images.

Entering the image names as command line input:

Most similar container image to the given image of an object - Weekly Report 2

Last week, I started the image processing part of the project and used SURF method which is implemented in OpenCV library (2.0). I prepared the proper environment to develop the sample code which was presented with OpenCV version 2.0.

What Does This Program Actually Do?

This week I had the opportunity of examining and improving that sample. The code begins with specifying parameters for extracting Speeded Up Robust Features using the line

Solution of file size problem in Demo

We cannot transfer large images in the first demo due to the problem of android operating system. A solution to that problem is the direct binary file transfer ability of org.apache.commons. fileupload.* package. The classes, FileItem, disk.DiskFileItemFactory, portlet.PortletFileUpload and servlet.ServletFileUpload, are used for binary file transfer by using HttpServletRequest and HttpServletResponse. We do not need raw char image transfer anymore because of servlet file upload technique.

1st Demo

Can you write down the things that you will do till next week here. (The things that we talked about.)

Weekly Report - Project Website

This week, I started to work on our project website. After exploring some alternatives, I decided on using Google Sites to build our website. Google Sites is a free-to-use facility from Google that makes it possible to build websites. Websites can be created from scratch or by using some predefined templates. I looked through the templates to choose the most suitable one for our purpose and found one.

On our website, there are pages that show the uploaded documents, tasks and actions to perform, a project timeline, our contact information and so on. It has all the features to coordinate and maintain the process of our project development. The site is located at https://sites.google.com/site/zephyrmetu/. Since we are given some disk space at the department to host our site, I redirected http://senior.ceng.metu.edu.tr/2010/zephyr/ to our website.

Finally, I thought of writing a piece of PHP code to maintain a log of the visitors of our page. Turns out, Google does not allow embedding PHP into web pages. So, I wrote the code to keep track of the visitors in PHP and uploaded it to our disk space at our department. Every time our page on Google Sites is requested, this script runs on the department's machine and returns the result. Google Sites allows this kind of "embedding" through a facility named "Embed Gadget". So, we have a log of visitors now.

Tolga AKIN
Department of Computer Engineering
Middle East Technical University

SVN -Weekly Report 1

First Task
This week, before starting to implement core functions of our project, I started to work on how to use Subversion in eclipse in order to control our source codes in a better way. As far as I see, using SVN via Eclipse is much more easier than using it from the command line.

First, we have to prepare our environment to use SVN in eclipse.
From Help->Install New Software tab, we will use “http://download.eclipse.org/technology/subversive/0.7/update-site/” site for downloading SVN.

OpenCV and SURF -Weekly Report 1

As we have mentioned in our project design, we are going to use image processing techniques to measure the similarity of two images. One of the methods we have choosen is SURF, and the detailed information will be found in here. Luckly, the method is implemented in OpenCV library version 2.0.

Server Side of the project

The server side of the system is a web service to be accessed by not only an Android application but also other type of applications. This feature makes the system more generic and usable. Apache tomcat and axis2 technologies are used for web service. Due to lack of web service support of Android operating system, a servlet is written for direct http request and response with the mobile application. This servlet forwards the message and the image to the web service.

Server Side of the Project (Admin Console)

In order to control the server side of the project, we decided to create a user friendly admin console. Currently it consists of web pages for admin and user tables but it will be improved during the development of our project. 

Client Side of the Project (Android Application)

During graphical user interface design, we took simplicity into account. Since functionality is the main purpose of the GUI design (the KISS principle), developing simple functions for user is what has been done so far.

Now, we will explain the details of the user interface of the Android application.

Application Entrance Point

The application will start by double clicking the icon presented to the user on the main menu and “Conqi” will be the user friendly abbreviation of the application name “Conqueror”.

Main Window

This is the menu of our application. The user will be able to take a picture, view the gallery consisting of the pictures she had taken via Conqi and see her location on the satellite view of the map.

Take Picture Window

This is the screen the user will encounter, when she has already chosen the “Take Picture” option. After taking the picture, she will see the still image of the landmark with information given below it. Additionally, she will be supplied with more options with menu button of the phone.

More Options Menu

If the user wishes to see other options and she has already pressed menu button, she will encounter with this screen. Through this screen, she will be able to get more information about the landmark that she has previously viewed. Furthermore, she may send the tagged picture to other mobile phone users. Additionally she may save the picture to her phone if she chooses “Save” option.

View Gallery Window

When user chooses to view the gallery, she will be able to see the pictures in the SD card of the phone which are taken by Conqi itself. The user will be able to display them and send a picture via MMS if she chooses an individual picture.

View Gallery Window Displaying an Individual Picture

While the user is displaying the gallery, if she selects an individual picture, she will face this screen. She will be provided with “Send as MMS” option which enables her to send the picture to someone else via MMS.

Locate Me Window

If the user chooses “Locate Me” option in the main menu, she will be able to see her location on the satellite view of the map. She will be able to wander around and and zoom in or zoom out in the map.

Mobile Conqueror

Mobile Conqueror is a system that can be easily used via mobile phones. Its purpose is to present the user with data about their surroundings, nearby landmarks, and other points of interest by overlaying information on the real­time camera view of a smart­phone.