Thursday, October 01, 2009

webcam snapshots and lotus/domino

Download FlexContacts(WebCamPics).zip
I was recently tasked with creating an application that can take a picture with a webcam and post it back to a server. The first tool that came to mind during the initial POC was flex/flash as it already has built in support for webcams. I ran through and did a google search to see if anyone else had gone through this and found a nice example on Adobe's Developer Connection (adobe sample). The sample from adobe was a great start. The next step was to tie it into a lotus/domino solution.
On the Flex Side the solution steps are:

  1. Add an mx:VideoPlayer control onto the stage and add some init code to bind it to the webcam.
  2. Create an mx:Image control to capture the frame from the webcam as a bitmap to review the captured image
  3. Encode the bitmap image to the desired type (PNG, or JPEG are currently supported encoders)
  4. Base64 encode the image data
  5. Post the base64 image data to the back-end Domino server and into a richtext field
  6. Have an agent consume the base64 data and create an image file attachment in the notes document

I've created an abstraction to get a handle on the picture using a factory class called PictureFactory. PictureFactory handles the capture and conversion of the image and returns an object implementing a Pic interface. Below are screen pics of the UI and the function takePicture which uses the picture factory to take the snap shot. You can find this function in the CamWrapper.mxml file under the notesui folder in flex builder, and the pic code is found in actionscript package.

Prompt to take the picture

Captured Picture in the Contact Form
(see CamWrapper.mxml)
private function takePicture():void{
//get an instance of PictureFactory (singleton factory class)
var factory:PictureFactory = PictureFactory.getInstance();

//get an instance of Pic (JPEGPic) which has a few different methods associated w/ it
//notice how we are passing in the camView (note it can be any UIComponent)
var pic:Pic = factory.createPicture(this.camView,PictureFactory.TYPE_JPEG);

//create a new DataXferEvent to store the picture object
var event:DataXferEvent = new DataXferEvent(DataXferEvent.EVENT_TAKE_PIC);

//store the picture object in our custom event;

//dispatchEvent sends the event along w/ the picture data out to any listeners.

Handles the DataXferEvent and binds the picture's bitMap to mx:Image to show the picture in the UI (see ContactForm.mxml)
private function bindPicture(event:DataXferEvent):void{

Responsible for posting the contact object to domino (see ContactForm.mxml)
private function postContact():void{
//build the parameter object
var params:Object = new Object();
params.pictureData = this.pic.base64Data;
params.eMail=this.eMail.text; =;;

var url:String = Cfg.buildUrl("fmContact?CreateDocument");

//if we have a unid in the formData XML then we use the SaveDocument else
//we use the create document.
if(formData!=null && formData.unid!=null){
url = Cfg.buildUrl("0/" + formData.unid + "?SaveDocument");

On The Domino Side
Now that we have our picture data ready to be posted, we need to have something in place on the Domino side to convert that base64 data into a JPEG again and attach the file to the contact document. Normally I'd use a java object to process the base64 data back to binary, but decided to look and see if there were LotusScript alternatives. I was very happy to find a nice little function in the NotesMimeEntity class called SetContentFromText that can convert base64 data back to binary. I created a single LotusScript class called PictureUtils containing one method to process the base64 data (see below code fragment)

Public Function processBase64Data() As String
On Error Goto MyErr

Dim session As NotesSession
Dim stream As NotesStream
Dim parent As NotesMimeEntity
Dim child As NotesMimeEntity
Dim item As NotesItem
Dim header As NotesMimeHeader
Dim picName As Variant
Dim fileName As String

fileName = picName(0) & ".jpg"

Set session = doc.ParentDatabase.Parent


Set stream = session.CreateStream

'get a handle on the picture data in the rich text item pictureData
Set item = doc.GetFirstItem("pictureData")

'write the text data into the stream
Call stream.WriteText(item.Text)

'create the new mime entity in the document
Set parent = doc.GetMIMEEntity("pictures")
If parent Is Nothing Then
Set parent = doc.CreateMIMEEntity("pictures")
End If

'create the new child
Set child = parent.CreateChildEntity()

'set the header information so we can name the file "picture.jpg"
Set header = child.CreateHeader("Content-Disposition")
Call header.SetHeaderVal({attachment; filename="} & fileName & {"})

'calling SetContentFromText will attach the file into the document.
Call child.SetContentFromText(stream, "image/jpeg", ENC_BASE64)
Call child.DecodeContent()

'remove the picture data as we don't need it anymore.
Call doc.RemoveItem("pictureData")

'save the changes.
Call doc.Save(False,False)


processBase64Data = fileName

Exit Function
Dim errBody As String
errBody = "Error = " & Lsi_info(2) & " " & Str(Err) & ": " & Error$ & " on line: " & Erl()
Call errLog.GenErrorLog("utils.PictureUtils",errBody,"")
Exit Function
End Function

One More Thing...
While I was creating this post, another thought came to mind. How cool would it be if a user experienced an error and in the exception handling a screen pic of what they are viewing at the time of the error is sent back to the server? This can be achieved by using the PictureFactory and passing in any UIComponent reference. I updated the code behind the "Test Error Log" button to capture an image of the current contact form when the exception is generated (see screen pic below).

Flex Error Document w/ Pictures attached

Picture captured using Test Error Log (in error handler)
That's it!
Below is a link to the .zip file containing both the flex project, and the .nsfs. You should be able to load the FlexContacts.nsf without issue to your domino server. Please let me know (via comments) if you run into any issues.


Gayatri Gauda said...

Can you tell me the otherway around?
i.e. I have embedded am image in a document by converting it to MIME using lotusscript NotesMime and NotesStream class. However, now I need to extract those image as attachments i.e filename.jpeg.

Can you help me with the soluton?

Mark Ambler said...

Try setting the session.ConvertMime=true to convert your doc to rich text, then use the doc.GetAttachment(attachmentName) method to get a handle on the NotesEmbeddedObject. There should be an example of this in the notes help, or you could take a look at the image resource import post I have on my blog

Mike Butcher said...

Is it possible to upload an Base64 encoded image to Domino Document and have it show as an image when you view the page in Domino? Do I need to Decode the image once it gets to the Server?
Please help..