JavaEye - EyeToy -like Control

Summer of Code

I've finished the learning of JMF stuff, quite funny and easy to use (compared using DirectX to access webcam). I decided to use chroma detection for separting the image of user and background because it is simpler and quite accuracy compared with motion and edge detection. (Although it means a simple calibration will be needed). On the other hand, Deron and paulby have just emailed me about adding Gesture Detection of LG into JayaEye . It sounds great. Anyway, there are many stuff needs to be done and i found that it is quite easy to out of scheduel I will hurry up !!!

Platform using: Currently Windows, using Netbeans 4.1.

Introduction:

Has anyone watched Minority Report? Have u seen Tom Cruise just use his hands moving in a 3D space to control the computer? Do you want to do it in the next generation Java Desktop? If yes, I can make it to become true.

Description and Explanation of the Projects Approach:

The philosophy of design is simple, because of the worrying of Sun on capability. It is better to use and stick on cheap/easy-to-buy device. My design uses WebCam as an input device to serves as complement or alternative for above design. Just like EyeToy , by using some sophistic algorithm, computer can distinguish users with background without using blue screen. Machines can react with user based on users’ movement gesture and touching (e.g. press button, move windows, etc). The idea is nothing new, there are many products using such technology. EyeToy is one of examples, another example is iCamPlay (http://www.balrogsoftware.com/icamlicence.html), iCamPlay is a library for motion, chroma and edge detection for webcams using Video For Windows, this library can be used not only for games, can be used for education, multimedia, bussiness, etc. (ref. iCamPlay Help Files). iCamPlay, however, is not open source and based on Win32 which cannot be cross platform (main advantage of Java) and limits its usage. In my project, I will build an iCamPlay-like library (JavaEye ) and apply it into LG3D . It is not difficult to do so because the algorithm is available. The concept of WebCam -based control and gesture monitoring has won Easy Finder - Trading Express Entrepreneur Contest (2004) and Category Awards in Health Care & Personal Devices of PolyU -IDT Innovative Entrepreneur Contest (2005). Thanks for JavaEye , users can easily to control LG3D in a Cyber 3D way, users can:

1.  Start / Close an Application
2.  Maximize/Minimize/Focus a running application
3.  3D movements for Windows
4.  Button pressing
5.  Virtual Keyboard stimulation
6.  Implementing an batch focus mechanism, e.g. Close/Minimize all windows
7.  Order user-define command based on hands gesture.
8.  Etc… Depend on the creativity of future developers

Benefits:

1.  Stick on cheap/easy-to-buy device.
2.  Users do not need to wear or attach/touch any devices, just move their hand or body.
3.  User can set hand gesture for faster program opening and command applying.
4.  Cyber and more 3D feeling, not just in 3D visual, but also 3D touch sense.

Deliverables:

Here are items which will be deliverable after the end of SoC :

1. Code: It is the main part of project. The code will be written in OO format and will be encapsulated as easy assessed API in order to facilitate the use of other developers.
2. Javadoc of Code: It served as Help file for Code. The javadoc will include the Hierarchy of different objects, the overview of members and functions inside objects and little description and usage method about them. So that other developers can use it as reference when using or modifying the code.
3. An executable prototype: It is as important as the code itself, so that mentor and other users can feel the usage of JavaEye instantly. It is also important for user behavior analysis later on so that we can know how users react with the JavaEye , and hence, how to improve and adjust it when there are something wrong
4. User behavior record: The behaviors and reactions of users will be fully recorded through video. The methodology of user behavior testing will also be designed and documented. It is useful because other developers can use it as reference and even insight for future development.
5. User document: A user document about the usage of JavaEye and FAQ will also be written in order to teach super-beginners to use JavaEye .

Schedule:

1. Use Case, interface design, OO design, code testing method design, learn lg3d dev, etc: 3 weeks
2. Implementation (Code writing) : 2 weeks
3. Test, debugging and adjustment: 4 week.

Projects experience:

Knowledge:

28 July 2005