OpenCV – Select, Capture And Save Camera Feed

EyeLine Video Surveillance Software

This Video Shows how to Capture and save the camera feed using OpenCV. It also Details how to select different cameras.

As always, To compare your scripts, if you have issues, I have included notes and the completed scripts at the bottom of the page which you can view or download from my Google Drive. The scripts also have notes as comments which appear after a #.

Remember if you can’t view fullscreen, you can right click the ‘youtube’ icon in the bottom right and select ‘view on Youtube.com’

Notes

The supplementary information can be found on the official OpenCV website https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_gui/py_video_display/py_video_display.html

The # is used in python to comment code after it on the same line. The importance of using this to comment code is the compiler ignores this when running the code but anyone coming later to work with your code can see why you have used the code you have or any notes you may have put in to modify it.

Some people say that “””comments””” is ok for multi line comments but this is wrong as it can get you into trouble. This is because it is a ‘docstring’ which is treated slightly differently. more info on ‘docstrings’ and their rules can be found here https://www.python.org/dev/peps/pep-0257/ but it is a slightly more advanced topic that you should save for later. Just remember only use a ‘#’ for comments on each line.

Download Links

Github OpenCV repository https://github.com/opencv/opencv/

Completed Scripts

Below is the link to the Google Drive repository for this lesson. You can either view online or download.

https://drive.google.com/drive/folders/190fHm-aJZfLM41rWjFPHxJycKteIq5zW?usp=sharing

OpenCV – Read and Write Images

Xara Web Designer

This Video Shows how to Open, Read and Write images using OpenCV. In the last post, I mentioned he flicks back to using Pycharm IDE for code editing so definitely go to https://www.jetbrains.com/pycharm/ and download / install the pycharm IDE which at present is free in community edition.

To compare your scripts, if you have issues, I have included notes and the completed scripts at the bottom of the page which you can view or download from my Google Drive.

Remember if you can’t view fullscreen, you can right click the ‘youtube’ icon in the bottom right and select ‘view on Youtube.com’

Notes

When copying lena.jpg, depending how your file structure has formed, when running the script, you may get an error. Try right clicking on the ‘venv’ folder and copy the image there or drag and drop from the main folder location. The script should now run and display the ‘lena.jpg’ image.

Keyboard key values can be found at http://www.asciitable.com/. It looks like below. Use the key ‘DEC’ column value. The key description is in red in the last column.

Ascii Table
http://www.asciitable.com

Download Links

You can download Pycharm from https://www.jetbrains.com/pycharm/

You do not need to but if you want to download microsofts awesome free code editor Visual Studio Code, This is the link https://code.visualstudio.com/ but all of the lessons are completed in Pycharm which I would advise using to follow along and then convert to VS code later when you are experienced.

Github OpenCV repository https://github.com/opencv/opencv/

Completed Scripts

So I hit another WordPress file type issue and can’t share script files so below is the link to the Google Drive repository for this lesson. You can either view online or download.

https://drive.google.com/drive/folders/190fHm-aJZfLM41rWjFPHxJycKteIq5zW?usp=sharing

Opening Downloaded Files

Sometimes your browser will not allow you to open downloaded files from the bottom of the browser, especialy if they are .zip files but you can either use the drop down arrow next to the downloaded file at the bottom of your browser and select ‘open in explorer’ or go to your ‘Downloads’ folder and find it there. Again if it is a .zip file then you can right click it and select ‘Extract All’ which will create a larger size folder version of the zip file. This will work now.

OpenCV – Installing on Windows

Xara Web Designer

If you are using a Windows Operating system i.e. Windows 10 then this is the video showing you how to install OpenCV which is crucial to learning. If however you are using a Linux Ubuntu system then please look at this video https://youtu.be/cGmGOi2kkJ4

The topics covered in this video are also useful for learning background features in Windows 10.

Please see the links and text below to follow along.

Download Links

You can download Python from https://www.python.org/

You do not need to but if you want to download microsofts awesome free code editor Visual Studio Code, This is the link but the rest of the lessons are completed in Pycharm which I would advise using to follow along and then convert to VS code later when you are experienced. https://code.visualstudio.com/

Commands for command prompt

  • python –version
  • pip –version
  • pip install opencv-python
  • python
  • import cv2
  • cv2.__version__

Opening Downloaded Files

Sometimes your browser will not allow you to open downloaded files from the bottom of the browser, especialy if they are .zip files but you can either use the drop down arrow next to the downloaded file at the bottom of your browser and select ‘open in explorer’ or go to your ‘Downloads’ folder and find it there. Again if it is a .zip file then you can right click it and select ‘Extract All’ which will create a larger size folder version of the zip file. This will work now.

Command Prompt

Windows 10 is what they call ‘evergreen’ which in laymans terms means it is always updating and changing. This is normally behind the scenes stuff but does affect how the menus work. At the time of writing, to get the command prompt in the method shown on the video, you have to

  • Right Click the Windows Icon (Menu Icon)
  • Select ‘Run’
  • Type ‘cmd’ in the box
  • click ‘OK’ or press ‘Enter’

OpenCV – Introduction

Xara Web Designer

OpenCV is all about processing Images from still photography or film and converting the data in the images to computer code and gives you the ability to work with the raw image data.

Thats a big piece to get your head around but I have found this set of youtube learning videos that starts at the basics of how computers see images and takes you through it in bite size pieces up to the capability of reading lane marking for self driving vehicles. I am also adding notes and supplementary notes after the videos to help understand as this site is geared towards knowing nothing and taking through to being capable yourself without needing to attend college. I myself am self taught on most things with the guidance and material provided by others on the internet or libraries. The downside is that you do not have a piece of paper for a job interview but you will have demonstrable skills so stick with it as you can learn anything.

The you tube video below gives a good introduction to OpenCV and how computers see images. It also includes an brief introduction to ‘numpy’ which is a highly optimised set of Python objects for working with images and numbers.

Please see the links and text below to follow along.

If you have never coded with Python and don’t quite follow along then please have a look at the following tutorial but if you just stick it out, you will pick it up. Think of coding like when you learnt to drive or ride a bike, scary at first but second nature after a while.

youtube – freecodecamp.org learn python in 4.5 hours

Autonomous Platform – The Intro

Building blocks of an autonomous mobile platform may sound futuristic but ever since I was a child, I dreamed of building robots, so I feel so fortunate to be alive and involved in science / engineering in this age as this is all possible on a budget and here is the start of my journey.

FastDomain Web Hosting $6.95

I started to look into learning Computer Vision as i want to build Rovers and Drones that are not only remotely operated but also aware of their surroundings for automated BIM capture to start as a commercially viable platform but with the insight in mind to expand these platforms for emergency service too once the core platform is established.

The mechanical aspects are an obvious hurdle which require knowledge of what role the platform will take i.e. Land, water or Aerial based. What work is to be done, what accessories etc. It all starts to take physical shape working back from the design brief / CtoQ’s / Goals or scope of the design vision.

Once you have the basic mechanical concept then Electrical components start to take shape to provide the mechanical structure with the motion it requires for the length of time required between charges and also the charging / automated battery swap criteria. I use battery swap loosely as when designing electric powered vehicles, I do not rule out liquid batteries which drain and fill electrically charged fluids rather than a solid lump of a battery.

Once the bulk of the electrics are designed, you can start placing the control electronics and sensing devices (camera,lidar, Ultrasonic, bump switch etc) but modifying the electrics to suit.

That just about sums up the overview of Robotics Hardware which for an engineer is not easy but not an impossible challenge either.

Now for control software we could start from scratch using Java ( which is not freeware for much longer) or Python ( which would be great) but for most standard platforms there already is Open Source Robotic Control Software (flight software) ready to be tweaked. For Rovers (land based) see Ardupilot.org and in fact this will do every vehicle type but for Aerial platforms, also look at px4.io or droncode.org as these are industry supported and in development. I will also mention that you need a compatible autopilot hardware kit, which for me with a raspberry Pi 3 will be the Navio2. These tend to come with a GNSS antenna for high location precision.

There is also numerous open source ground control and mission software like QGroundControl.com using MAVLINK or Mission Planner. If using a PC, you will need a telemetry transmitter and receiver kit(433 or 868MHz for UK, 900 for USA and Canada). There are numerous free offering for tablets too in Android or IOS flavours.

Ok So we no have the blocks to create a fully functioning remotely controlled semi autonomous vehicle but how do we make it autonomous. Well that, ideally would take LiDAR and Computer Vision with OpenCV.

LiDAR is an option at this point but with limited open source options we will leave this for a more advanced Robot ‘The Mark 2’.

So lets talk about Computer Vision. This is the route that Car manufacturers are going down, with support of LiDAR, and is all about detecting dangers and picking out data from the camera and turning that image data into usable sensory information that can be processed by the Controller. To do this we can use a piece of open source software called OpenCV. I will mention that OpenCV will also process LiDAR data so we can expand the capabilities later.

This will turn the image data into code which we can interact with using Python code.

At this point I will mention that I am not going to create videos on how to use OpenCV myself because I have found an abundance of youtube courses which are perfect so why remake the wheel. Instead I am going to compile pages with other peoples videos and supplementary information to help cover all bases. This means that I can write faster and other kind people who have taken the time to create content get the boost from more views on their videos.

I will however embed the videos and also give the code from the video which I have tested and added comments – yes we are all doing this together.

So without further a do, lets get on with learning OpenCV and really get our robotics alive. Once we understand how this all works, we will come back to the design brief capabilities and then hopefully on with the design and build. This should be fun, as is most blue collar engineering, but please share the posts with colleagues / friends and comment back with suggestions. Don’t forget you can always email privately at contact@vulcanrealm.com.

If you are looking for all the current lessons then please look under this page’s dropdown in the navigation menu to the left.

EyeLine Video Surveillance Software

POLICING AND BIM – A WIN WIN PROPOSAL

With ever increasing pressures on policing budgets for more Officers on the beat and such an overwhelming task for utilities and businesses to get BIM Level 2 and PAS 256 compliant, there is the potential for a partnership that will benefit Private industry with an expandable framework for the police and public services.

FastDomain Web Hosting $6.95

I will start by asking you to be open minded and dismiss any BIG Brother concepts or conspiracies and look at the potential for good. There is nothing wrong with technology, only those that use it for bad.

I recently posted a short piece on 360 video for photogrammetry so we can model the travelled world by utilities giving a targeted group of selfie customers some 360 cameras and free hosting with the proviso that the utilities could extract the built environment data. It can be found here – upgrade to 360 video cameras. This expands on that, but changes the operating user for better utilisation of funds.

So as discussed in the previous article that pointclouds and models can be created from the 360 videos which can be uploaded to a central server and post processed to identify things like telegraph poles and manhole covers which will give point to point locations of sewers, and buried telephone and power lines as well as other services. This is nothing new but the software and algorithms get better every day.

So why are we not doing this already – we are but it costs a fortune in labour and meetings to target the priorities of that small survey labour force. This is crazy as we have to get it done anyway and it is always the end customer that has to pay for it in the long run, whether it be taxes or recovered through the bills of the end product like SMART meters or did bills just go up at the same time by coincidence – I am not on the inside on that one, there was a report by BEIG but I refer you to the UK parliament paper to research yourself, on how they are not as good as they are made out, and there are the issues of electro smog (see the difference between radio emmission limits between the west and east + switzerland) and once you have one – they wont remove it – my personal experience is the same with my energy provider.

Ok back to topic, So we have 2 issues – Companies not currently investing enough in labour or infrastructure to do this and multiple organisations duplicating labour. What other issues do we have – One large one is an underfunded police force. Hey lets partner with the police to capture the imagery as they go everywhere and repetitively so it is always up to date.

I know what you are saying – Big Brother State, Hate to burst the bubble but we already are there technologically and it is only the control measures and procedures that stop a 1984 distopia – and there is no reason for this to change especially in the UK.

Unhackable Kaspersky

The technicalities of the police capturing 360 footage can be as basic as a helmet or selfie pole (attached to uniform) mounted 360 camera or even go the whole hog and mount an Infra red camera directly above the 360 so maximising the opportunity.

This will give the benefit for capturing the built environment for BIM and PAS 256 but also the police forces benefit from 360 recording and reporting of police activities.

So why would this benefit the police in the long run. Well before we get to the technical advantages, there is the fact that money would be diverted to the police forces as a surveying service / data source which would be an easy sell to private industry as they are omni present and thus have the resources ready to go now without immediate investment in staff, but, this increase in finance would aid the police to increase its staffing levels to provide a better policing service and more up to date data source which benefits the whole of society and public purse as you move the policing sector from being an economic drain on the public purse to a revenue generating / self supporting sector which gives it a positive business investment plan.

This product can then be used for the police service’s own gain by creating Mixed reality software/hardware that aids an officers automated threat detection system, much like technology applied to military aircraft – something basic to start like someone about to attack from behind should not be too hard to incorporate, thus giving health and safety benefits to police officers on the beat. Having just mentioned the military – Just think of the benefits of this same technology to peace keepers, maybe they have something already they could share to aid this development.

The Most Affordable Meetings On The Market

So expanding on this, as an Agile development, you can develop the ability to combine a localy processed geolocated video stream to help locate geographics and comparitive video overlap to detect what has changed over time. This has its obvious law enforcement and military applications, but now if we extend its application to search and rescue, Fire Service, Environment Agencies – we have a full cross sector data reference system for a virtual world construct that can automatically detect areas of risk or interest depending on the criteria. Imagine being able to direct flood rescue personnel to search certain areas with a geographic location and image on their helmet display of what they expect to see underfoot in real time.

I know people are saying its too heavy and will not work but we can combine a micro computer (think Raspberry Pi) with different operating systems i.e. Android / Win10IOT, that support something like openCV so we can pre load areas onto it in a mesh format which is much lighter than pointcloud so we are not far off technologically thanks to opensource gurus with ethics.

Ok, I have strayed from the original point with good reason – Whenever I talk about Virtual world I come up against – Who will use it? and if I answer Future growth planners, maintenance, Construction – I always get the answer of they already do something and its their budget which creates a great deal of pain for me as the public or customer pay for everything in the end so I look at the best efficiency for them. Yes we need to combine this or tailor to application, and I do not have all the technical answers, maybe the guys that came up with OpenCV and mapping companies like ESRI do, but this is just a concept to be enhanced. If we all worked together in a cross sector way for society, which we all are members of, rather than self image or local profit / reward, we might get somewhere – He sais knowing he will place ads in this post.

So moving on with the Police service benefits, you can incorporate feature recognition and OCR (Optical Character Recognition). The Feature Recognition aspects may be as low key as identifying a truck or car for traffic enforcement but could be as enhanced as identifying persons of interest from the crime database. Some algorithms are as advanced as to detect behavioral conditions which could help dealing with people on a friday night or, being 360 degree vision, be able to detect someone needing help in the distance behind as the officer may be distracted with a lower priority task.

The OCR feature could obviously help track vehicles but also to report an officers location to control which will help in built up areas where satellite navigation systems have reduced accuracy.

Ok, with feature recognition software, you also get facial recognition so moving away from the beat policing, If a crime had been committed and you had officers walking the BEAT as it were, the software could pick out regular faces who may provide leads on past cases, not just recent but if you think of transference of geographic habitual habits, i.e. someone at 18 Dortmund street always buys a pint of milk at the corner shop of never was ere lane, then 30 years ago, the resident of that address is likely to of done something similar which may help gain witness leads to close cold cases.

One other point to consider on recent crime locations is the video would provide Accident / first responder investigation – never miss a thing even if someone removes evidence later. As long as someone was there – you have a full record of the crime scene. This would also aid other services like the AAIB when they fly out to plane crashes in the middle of densely populated forests. The software could even be tailored to search areas instead of relying on the skill and attention of the searcher. Imagine the time it would save identifying the four corners of the aircraft if you could send locals with a hat on before the pillaging starts.

Ok, I think I have nailed why the police should be considered the geographic photogrammetric surveyors of the world – if you agree then please share this post on social media and get them the funding to police society properly.

I wanted to share this last note to those that have read this and are scared of too much technology in the states hands. I had a conversation with someone the other day, and as much as we disagree on so many things, we both agreed that the way to avoid an autocratic ‘Big Brother State’ is to keep the ‘Bobby on the beat’. The simplest way to achieve this is for us to all make them an economic benefit to the state as industry invests in profit channels and I believe this is one without corrupting the law enforcement high standard, just needed some high level lateral thinking…

Now imagine if we equiped several police vehicles with high speed laserscanners above the light bar – well maybe you should read https://vulcansrealm.com/2019/04/14/driverless-autonomous-vehicles-how/

On the other hand if i am in jail next week – the system is corrupt – No freedom of speech – its a conspiracy and I am innocent !!!!

TEXT EXTRACTION WITH FreeOCR

Optical Character Recognition software has been around for a while now and is used in many applications from number plate recognition to document scan to text. The big opportunity comes when incorporated into Surveying and robotics. For this, many companies turn to big Cloud Computing Products such as Google Cloud or Azure but there is a cheaper way if you are prepared to do some computer legwork and are not after a one stop shop solution. If you want one stop then GCP or Azure is the place to go but you will pay for it.

Rescue deleted files or photos - O&O DiskRecovery

Alternatively you can combine smaller products as below to achieve similar great results for efficiency in mass processing. I have remained generic and not to mention brands as either the open source or proprietry products will return good results.

The big hold up has been getting GOOD free opensource/freeware OCR software. We now have it in the form of FreeOCR downloadable from paperfile http://www.paperfile.net.

FreeOCRIcon

This software allows you to scan all documents into word format extracting the text automatically and works with pictures too. It uses the tesseract OCR engine which is at the following GitHub page http://code.google.com/p/tesseract-ocr/ and can be compiled into your own software creation for the aspiring coders amongst us.

Lets say you want to scan CCTV footage for registrations of people coming and going through a gate, Simply turn the video into images (maybe using VLC) and load the saved images into FreeOCR. Hit the OCR button and it will convert any text in the images to a text file.

This might be a good idea for automated scanning of CCTV footage after a crime to find witnesses.

Another alternative use would be for BIM and scanning asset tags or data plates. Lets say you have some georeferenced images taken with something like a SPECTRA SP20 you would be able to cross refer the OCR recovered Model and Serial number with the GeoTag in the images Metadata in an automated way to geolocate the asset data in the database.

You would already have to have an asset database to query but  you could add assets like this too.

Maybe you could automate cheaply using a GoPro Hero5 (or later) set to Linear or Medium Field Of View, 50 Frames Per second and good forward lighting. You would also need a piece of software that uses tesseract and will record the frame or picture number against the OCR output and image metadata including GPS data.I mention this method with a GoPro example as you could use the same video footage as you take for photogrammetric modelling where the results would improve with the better camera and also records the Geolocation. If you need a GoPro, they now supply the Hero7 12MP on Amazon. Click the image below to see the listing and the Specs.

This Photogrammetric modelling would provide the basis for the 3D virtualised world engine for self operated robots later.

Now Moving on from data collection, imagine if robots could read languages and understand, orientate and operate themselves with OCR or feature extraction with 360 cameras, then we are talking that we are close to robots automated operations in changing environments.

Now lets combine other work with driverless cars and the virtualised world engine, we are talking about fully autonomous vehicles or self operated machines in a variable world.

The helps the Future to be Exciting as we change how we apply current technologies to deliver futuristic capabilities today.

Insta360 Air (Micro USB) 360 Camera