Category: Web application dev

[Paper published] Check out our new (deep) machine learning paper for flood detection

New machine/deep learning paper led by Liping: Analysis of remote sensing imagery for disaster assessment using deep learning: a case study of flooding event . 

A full-text view-only version of the paper can be found via the link: https://rdcu.be/bpUvx.

 

Check out this page for Liping’s more publications.

 

[django] Sending matplotlib generated figure(s) to django web app

This post introduces how to serve the figures generated by matplotlib to django web app without saving on the server.

In your django views py file, import the following libraries

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

from io import BytesIO
import base64

Note: the two lines of code above in blue need to be placed at the very beginning of the py script; otherwise, you would meet the following error: _tkinter.TclError: no display name and no $DISPLAY environment variable.

In the views py file, in the function that you defined to pass the image data to the front end template file, add the following code:

buf = BytesIO()
plt.savefig(buf, format='png', dpi=300)
image_base64 = base64.b64encode(buf.getvalue()).decode('utf-8').replace('\n', '')
buf.close()

Note: in the function that you defined to pass the image data to the front end template file in your views py file, remember to send the value of the variable image_base64 via, for example, json.

Now, in your front end template file, you can add the following image tag.

<img src="data:image/png;base64,{{image_base64}}" alt="some text to display to your users when the image does not show correctly" width=500 height=auto />

You now should be able to see the figure displayed on your web app page:)

For more details about using data url to pass image data to front end html file, check here (pdf).

 

 

List Node process and kill specific process if needed

This post provides instructions on how to list running nodeJS application, and  how to kill specific process.

  • List node process:
$ ps -e|grep node
  • If you want know, how many nodejs processes running, you can use the following command
$ ps -aef | grep node

It will give list of nodejs process with it’s project name. This is helpful when you are running multipe nodejs application & you want kill specific process for the specific project.

  • Kill  specific process using the following command:
$kill -9 XXXX

XXXX is the process number to be killed.

  • You can kill all node processes using 
$ pkill node

If all those kill process commands mentioned above don’t work for you, probably you will need to check if you were using any other packages to run your node process.

For example, if you were running your node process using PM2(a NPM package), the kill [processID] command indeed will disable the process but it will keep the port that you ran the nodejs app occupied. You will  need to go into PM2 and dump all node process to free up the port again.

 

You can use the following comment to check all running process, including node and java apps etc.

$ ps -ef

you will see a list with the headers as the following line:

UID   PID   PPID   C   STIME   TTY   TIME   CMD

 Among the line,

STIME means the time your app starts,

TIME   means total CPU usage of the corresponding process

and

CMD is the app’s name running (i.e., name of the process, including arguments, if any)

 

if you only want to list node process,

use the following command,

$ ps -ef | grep node

if you only want to list java process,

use the following command,

$ ps -ef | grep java

You got the idea…

-e and -f are options to the ps command, and pipes  (i.e., | ) take the output of one command and pass it as the input to another. Here is a full breakdown of this command:

ps – list processes

-e – show all processes, not just those belonging to the user

-f – show processes in full format (more detailed than default)

command 1 | command 2 – pass output of command 1 as input to command 2

grep find lines containing a pattern

processname – the pattern for grep to search for in the output of ps -ef

So, the following command means:

look for lines containing processname in a detailed overview/snapshot of all current processes, and display those lines

ps -ef | grep processname

 

Commonly used commands for Node.js (Ubuntu)

This post provides some commonly used commands for Node.js

  • Start a Node.js application
$node server.js

Note: After you start your node application, go to the host name address (e.g., example.com) of your server and then a semicolon, and then give port name. (For example, if it is running on port 8081, then type the server host name address:8081/) then you should see your node application web page.

  • Stop a Node.js application
$Ctrl + C

 

 

Install Node.js on Ubuntu 16.04 LTS

This post provides instructions about how to install Node.js on Ubuntu 16.04 LTS. See this post for Node.js resources. (Node.js Offical Github Repo.)

NPM (Node Package Manager) is the default package manager for the JavaScript runtime environment Node.js. NPM hosts thousands of free node packages. In general, NPM is installed on your computer after you install Node.js.

There are several ways to install Node.js on Ubuntu:

  • Method #1 (our choice in this tutorial): Install Node.js with Node Version Manager (NVM) to manage multiple active Node.js versions

Using nvm, we can install multiple, self-contained versions of Node.js, which will allow us to control our environment, and get access to the newest versions of Node.js, but will also allow us to keep previous releases that our applications may depend on. (nvm is just like Virtualenv in Python, if you are familiar with it, which allows us to install multiple version of the same Python library into “virtual folders” by pip.)

This is the method we will cover later in this tutorial.

  • Method #2: Install the bundled Distro-Stable Version Node.js (version 4.2.6) – it is very simple to install, just one or two commands.

Ubuntu 16.04 contains a version of Node.js in its default repositories that can be used to easily provide a consistent experience across multiple systems. At the time of writing, the version in the repositories is version 4.2.6. This will not be the latest version, but it should be quite stable, and should be sufficient for quick experimentation with the language.

This tutorial picked the Node Version Manager (nvm) based method, because it is much more flexible.

See below for the step by step instructions. (Check out the reading list below if you need the install instructions for other methods listed above.)

Step 0: (Before we get started) Remove old Node package to avoid conflicts

Open a terminal (Ctrl + Alt + T), and type the following command. 

$ dpkg --get-selections | grep node

# If it says install in the right column, Node is on your system:
#ax25-node                                       install

#node                                            install

Step 1: Install prerequisite packages

We’ll need to get the software packages from our Ubuntu repositories that will allow us to build source packages. The nvm script will leverage these tools to build the necessary components.

First, we need to make sure we have a C++ compiler. Open a terminal window (Ctrl + Alt + T) and install the build-essential and libssl-dev packages. By default, Ubuntu does not come with these tools — but they can be installed by the following commands.

$ sudo apt-get update

$ sudo apt-get install build-essential libssl-dev

Step 2: Install nvm

Once the prerequisite packages are installed, we can install and update NVM(Node Version Manager) using cURL. (Note: to get the latest installation version link, on the page scroll down to “Install script”.) 

$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash

Inspect the installation script with nano:

$ nano install_nvm.sh

#Note that we DO NOT need to add anything in the opened nano text editor window. We just need to create the .sh file. 
# Use Ctrl+O to save the file, and then hit Enter, and then Ctrl +X to close the file.

Run the script with bash (Note that run the following command in your terminal):

$ bash install_nvm.sh

It will install the software into a sub-directory under our home directory at ~/.nvm. It will also add some necessary lines to our ~/.profile file in order to use it.

To have access to the nvm functionality, we need to source the ~/.profile file so that our current session knows about the changes:

$ source ~/.profile

Now that we have nvm installed, we can install isolated Node.js versions.

Step 3: Install  Node.js

The following command will tell us which versions of Node.js are available for us to install:

$ nvm ls-remote
Output
...
    v6.10.3 (Latest LTS: Boron)
 v7.0.0
 v7.1.0
 v7.2.0
 v7.2.1
 v7.3.0
 v7.4.0
 v7.5.0
 v7.6.0
 v7.7.0
 v7.7.1
 v7.7.2
 v7.7.3
 v7.7.4
 v7.8.0
 v7.9.0
 v7.10.0

The newest version when I write this post is v7.10.0. We can install it by the following command:

$ nvm install 7.10.0

By default, nvm will switch to use the most recently installed version. We can explicitly tell nvm to use the version we just installed by the following command:

$ nvm use 7.10.0

When we install Node.js using nvm, the executable is called node (NOT nodejs that you may see in other tutorials). We can check the currently used Node and npm version by the following commands:

$ node -v  
# OR 
$ node --version

# Output
# v7.10.0


$ npm -v
# OR 
$ npm --version

# output
# 4.2.0

Step 4: using nvm to manage different versions of installed Node.js 

If you have multiple Node.js versions, you can see what are installed by the following command:

$ nvm ls

To set a default Node.js version to be used in any new shell, use the alias default command

$ nvm alias default 7.10.0

# This version will be automatically selected when a new session spawns. You can also reference it by the alias like this:

$ nvm use default

To learn more  about the options available to use with nvm, run the following command in your terminal:

$ nvm help

Step 5: using npm to install Node.js modules

Each version of Node.js will keep track of its own packages and has npm available to manage these.

We can use npm to install packages to the Node.js project’s ./node_modules directory by using the normal format. For example, for the express module:

$ npm install express

If you’d like to install it globally (i.e., making it available to other projects using the same Node.js version), you can add the -g flag:

$ npm install -g express

This will install the package in:

~/.nvm/node_version/lib/node_modules/package_name

Note that installing globally will allow us to run the commands from the command line, but we will have to link the package within a project in order to use it in that project:

$ npm link express

 

References and further reading list:

This tutorial covers two methods:

Method #1: Install the bundled distro specif Node.js version 4.2.6

Method #2: Install the latest version of Node.js version 6.x or 7.x

This post is very good — it covers the following ways to install Node.js:

Installing using the Official Repository
Installing using the Github Source Code Clone
Installing using Node Version Manager (NVM)

It covers How to Install Multiple Nodejs version with NVM and also covers how to remove Node.js 

This tutorial covers:

  • How To Install the Distro-Stable Version for Ubuntu
  • How To Install Using a PPA
  • How To Install Using NVM

there are a quite a few ways to get up and running with Node.js on your Ubuntu 16.04 server. Your circumstances will dictate which of the above methods is the best idea for your circumstance. While the packaged version in Ubuntu’s repository is the easiest, the nvm method is definitely much more flexible.

This is a pretty good tutorial. It covers 4 Ways to Install Node.js on Ubuntu. There are several ways to do this, but The author of this post recommended  Option 1: Node Version Manager (nvm). Here is the full list of options: