Science and Technology – Imaging

Confession

Anyone who has been in the IT fields as long as I have (30+ years) will tell you that they have more obsolete certification than active certifications. Personally I don’t see anything wrong with that, for in any field continuous education is important. Before I begin this article, I need to confess that this article is more for me than for you. You see it is going to get very technical for the average reader. So you may want to stop now. But if curiosity is your fore-tee, then please indulge yourself.

Now when I say this is technical, it is, but it is not complicated. Some background is in order. Ever since I can remember I have always asked the question why? If someone told me that this was not possible I would say why? I recall the time in university when we were using CCD (Charged Couple Device) cameras to measure star light. They were rudimentary back in the early 1980’s. With a 1024x768 resolution,you could get one for nearly $10K. So when everyone was cooling these things down to -78.5 C, the sublimation temperature of Carbon Dioxide to increase the signal and reduce the noise (S/R ratio) I was devising methods and pushing technology to operate at -196 C, the boiling point of Nitrogen, and when I got to that point attempted to get to the melting point of Nitrogen -210C. Because, you see, there were two reason for doing this. The colder you go the better your S/N ratio and the better your imaging, and secondly why not.

By the way, some may ask why LinkedIN? Two reasons, first it’s a good repository and I will find it when I need it. Secondly, there just might be someone out there that will find this useful. Be it a CEO, or a high school student itching to get a A in their science fare project. And with all the software being free! That’s just BONUS!

With this all said, I now welcome you to grab yourself a cup of JAVA, sit down and come on an adventure where we will look at measuring the the length of and the behaviour of a 24 hour period for March 7, 2019 as we look skyward.

The Hardware

To begin with you need some hardware. In this case a computer and a webcam. I am using an older tower with 8 usb ports with 4 GiB of RAM, a Intel Core i5 CPU 760 running at 2.80GHz x 4. It is a 64-bit Ubuntu 18.04.2 LTS operating system with 3 TiB of disk space. (Linux 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux)

The webcam is a Logitech QuickCam Orbit AF 2.0M Effective Pixels USB

The Software

Like any good experiment you are going to need some really good software. In this case I am using the following:

• GUVCVIEW – Standing for GTK+ UVC Viewer is a free software webcam application for the Linux desktop, started by Paulo Assis in 2008. The application is written in C. Guvcview is compatible with all V4L2 camera devices, using the Linux UVC (USB Video Class) driver and based on luvcview for video rendering. Originally made for the Gimp Tool Kit it has become the default Gnome’s toolkit. More on Guvcview later.
• GIMP – Standing for GNU (GNU is not Unix – put smiley face here) Image Manipulation Program is a freely available open source application for creating and manipulating graphic images that runs on Linux, other Unix-based operating systems, and also on Windows and Mac OS X, It is very powerful and although somewhat intuitive, under the hood will take years to master. With that said, there is a great community out there to help you along!
• ImageMagick-7.0.8-32.x86_64.rpm is a free and open-source powerful software suite for displaying, converting, and editing raster image and vector image files. It can read and write over 200 image file formats. Created in 1987 by John Cristy when working at DuPont, to convert 24-bit images (16 million color) to 8-bit images (256-color), so they could be displayed on most screens. It was freely released in 1990 when DuPont agreed to transfer copyright to ImageMagick Studio LLC, still currently the project maintainer organization. This application is not offered in the Ubuntu repository so to Install this package refer to https://linuxconfig.org/how-to-install-imagemagick-7-on-ubuntu-18-04-linux. We need this package to utilize the mogrify utility to do the cropping.
• PYTHON – No it’ s not a cup of JAVA, and no it is not a a large heavy-bodied nonvenomous snake wandering throughout the tropics, killing prey by constriction and asphyxiation. Although that may be the case, it really is a a high-level general-purpose programming language. Created by Guido van Rossum and first released in 1991, the programming language has come a long way, with API’s and other features you can find it everywhere. With my strong background in networking and server / storage, Python has become the language of choice in everything from networks to cloud based applications. I’m going to use it for image manipulation.
• R – is nothing new. A programming language first developed by John Chambers back in (this is right) 1976 at Bell Labs! This statistical programming language has not changed much, but with the help of John and Ross Ihaka and Robert Gentleman at the University of Auckland, New Zealand, and currently developed by the R Development Core Team (of which Chambers is a member).Yes it’s all in the name, organically called S (For Statistics), the resurrection of this forgotten language was named R after the first names of the first two R authors and partly as a play on the name of S. The project was conceived in 1992, with an initial version released in 1995 and a stable beta version in 2000. From there all hell broke loose as statistics once again became mainstream with Machine Learning and Artificial Intelligence.
• LibreOffice Calc – Well any spreadsheet will be useful. To tell you the truth I am using it because I’m lazy. Maybe I will incorporate this requirement in Python at some point, but sometimes when you want to do something fast, a good old spreed sheet just works. More on that later.

The Approach

Before proceeding it is good to have some basic Linux knowledge, like starting a program, opening a terminal and working with a terminal.

Step 1: Collect those images

• Assume your computer is powered on (Place smiley here)
• Connect your webcam to a usb port
• Power up GUVCVIEW to make sure that everything is working as planned.
• In the photo tab choose a name and location for the images. (Example: sky)
• Choose to increment the images or else you will just be overwriting them (Yikes!)
• Once you are satisfied power off GUVCVIEW.
• Open up a terminal session
• type in guvcview -t 900 (In my case I choose 900 seconds which equates to 15 minutes), you can choose whatever you want. (60 x seconds gives you minutes!)
• Some warnings here
• You will see that the caption on the camera says Stop. Don't press the button unless you want to stop it.
• You can control the panning and tilt now
• Don’t be alarmed with the message “V4L2_CORE: Could not grab image (select timeout): Resource temporarily unavailable”. It’s on elapsed time so the first picture won’t happen until your time is up, that is 900 seconds for my case and ever since after.

Step 2 – Preparing your images

• I like to copy my images to a secondary area. After all it took you 24 hours to collect them and you don’t want to loose them.V4L2_CORE: Could not grab image (select timeout): Resource temporarily unavailable
• Open up some images and determine the best place to crop the image. In my case I look at the area of the sky that has no sun.
• Using GIMP
• Open one of the images in GIMP.
• Using the Rectangle Select Tool (hotkey “R”), select the area you want to be cropped.
• Note the X, Y, Width and Height values GIMP gives you (have a look at the picture, you can find them in the GIMP main window).
• Close GIMP (or leave it open if you plan to use it again soon).
• Now working in the terminal. cd to the directory where the images are located. Then, let’s start with the mogrify command. The syntax for cropping is as follows:
• mogrify -crop {Width}x{Height}+{X}+{Y} *.jpg
• The {Width}, {Height} is your size and {X}+{Y} is your position values you got from GIMP! Note that I used *.jpg file, we recall mogrify is able to handle over 100 image file types.
• Fill in the values from the screenshot, the mogrify command would look like this:
• mogrify -crop 144x200+262+299 *.jpg
• hit enter and all your images will be processed as cropped

Step 3 – Process and get results

• You may have some images with 0 bytes, delete them they are of no value.
• Next run Python (I use PyCharm as my editor)
• The script I use is

from scipy import misc

from datetime import datetime, timedelta, timezone

import os

rgb = [3]

wrfile = open("/home/family/Pictures/Cropped/sky.csv", "w")

print("time, filename,red,green,blue,value", file=wrfile)

epoch = datetime(1970,1,1,tzinfo=timezone.utc)

for filename in os.listdir('/home/family/Pictures/Cropped/'):

if filename[-4:] == ".jpg":

x = os.path.getmtime('/home/family/Pictures/Cropped/' + filename)

modified = epoch + timedelta(seconds=x)

rgb = misc.imread('/home/family/Pictures/Cropped/' + filename).mean(axis=(0, 1))

print(modified, ",", filename, ",", rgb[0], ",", rgb[1], ",", rgb[2], ",",rgb[0]+rgb[1]+rgb[2], file=wrfile )

wrfile.close()

print('done')

• Run it!
• I then open LibreOffice Calc and perform the following tasks
• Copy the file name
• Paste it somewhere free
• Text to Column the “.” as a separator to remove the “jpg”
• Text to Column the “.” as a separator to remove the “-”
• The numbers represent in my case 15 minutes (60*15) so multiply by 15 and place in time
• Then copy the time column and then paste as numbers only
• Cleanup you spreadsheet. Should only have the following columns (time,filename,red,green,blue,value). Note the value is the addition of red,green and blue and represents the grey scale of the image (value = max(r,g,b))
• Insert column to left of time and call it actual
• Take note of the modified time of the first image located in your webcam folder. This is your starting point. In my case it is March 6, 2019 20:15 CST
• In my case the actual time will be saved as the formatYYYY-MM-DD HH:MM with the equation (=+J$1+(B2/1440) where j$1 contains 2019-06-06 20:00:00 as my starting point
• Now fill the equation to along the actual column until the end is reached
• Copy the actual column and then past special date and time only. Make sure you did not delete the actual title. If you did put it back.
• Now delete all those unwanted columns. Remember you only want to see (actual,time,filename,red,green,blue,value) as columns
• Now save the file as a csv file
• Run Rstudio (R version 3.4.4 (2018-03-15) -- "Someone to Lean On")

Import the libraries

library(readr)

library(ggplot2)

library(mgcv)

Then run the following:

sky <- read_csv("Pictures/Cropped/sky.csv", 
       col_types = cols(actual = col_character()))
View(sky)

a <-sky$actual
x <-sky$time
r <-sky$red
b <-sky$blue
g <-sky$green
v <-sky$value

plot(x,r, col = "red", main = "Red Band", xlab = "Time (min)", ylab = "Red")
plot(x,g, col = "green", main = "Green Band", xlab = "Time (min)", ylab = "Green")
plot(x,b, col = "blue", main = "Blue Band", xlab = "Time (min)", ylab = "Blue")
plot(x,v, col = "grey", main = "Grey Value", xlab = "Time (min)", ylab = "Total")

mydf <- data.frame(v,g)
model <- lm(g ~ v, data = mydf)
summary(model)
plot(model)

gm <- gam(g ~ v)
summary(gm)

ggplot(sky,aes(x,r))+geom_smooth()
ggplot(sky,aes(x,g))+geom_smooth()
ggplot(sky,aes(x,b))+geom_smooth()
ggplot(sky,aes(x,v))+geom_smooth()

The Results

gm <- gam(g ~ v)
summary(gm)

Family: gaussian 
Link function: identity 

Formula:
g ~ v

Parametric coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 0.0358860  0.3549782   0.101     0.92    
v           0.3426825  0.0009572 358.006   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


R-sq.(adj) =  0.999   Deviance explained = 99.9%

The above Gaussian identity explains 99.9% of the deviance which suggest a pretty good fit thus the following graph is meaningful.

The graph shows the camera picking up the light and first notable light is at 6:19 am with the last notable of light at 7:00 pm. This is not daylight as according to https://www.timeanddate.com/sun/canada/winnipeg March 7 sunrise was 6:59 am with a sunset at 6:20 pm giving 11:20:57 of daylight. The extra light is taken into account through the camera’s ability to effectively capture twilight.

Conclusion

Science and technology is has come along way, but the basics is still important. We have seen S built in 1970’s has become R in the 1990’s and has become mainstream as Machine Learning and AI moves forward. In this little exercise we push technology a bit and can even more it forward more. Looking for outliers in the observations we may be able to determine anomalies such as a plane crossing our field of view.

For More Information

If you want to get a better look, see it on my YouTube channel. https://youtu.be/pRchAODk-nc