Skip to content

wattsworth / puppet

Go to a project
Commit 88fb7f78 by John Doe
Options

working on docs

parent 43665687
Inline Side-by-side
Showing with 302 additions and 89 deletions
.gitignore
*~
docs/_build/*
\ No newline at end of file
docs/_build/*
.DS_Store
docs/conf.py
......@@ -30,7 +30,10 @@
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.githubpages']
extensions = ['sphinx.ext.coverage',
'sphinx.ext.imgmath',
'sphinx.ext.viewcode',
'sphinx.ext.githubpages']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
......@@ -90,9 +93,18 @@ html_theme = 'alabaster'
#
html_theme_options = {
"logo": "logo.png",
"show_powered_by": True
}
html_sidebars = {
'**': [
'about.html',
'navigation.html',
'relations.html',
'searchbox.html',
'donate.html',
]
}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
......
docs/configuration.rst 0 → 100644
Configuration
=============
In most situations Wattsworth should work well with the default settings.
For more complex setups involving systems that may loose power, are connected
through networks, or other special cases the reference material in this section
shows how to fine tune the installation to meet these needs.
Storage
-------
Make sure you set up data journaling on the nilmdb partition. This will prevent
data corruption if the computer looses power without properly shutting down.
Edit ``/etc/fstab`` and add the ``data=journal`` option to the parition with the
nilmdb database.
.. code-block:: bash
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
UUID=XXX / ext4 errors=remount-ro,data=journal 0 1
# add this --^
If the partition is the root partition you must add this option directly to the volume
as well, otherwise the system will not boot properly. For example if your
root partition is on ``/dev/sda2``:
.. code-block:: bash
$> sudo tune2fs -o journal_data /dev/sda2
<span class="lead" id="storage">Storage Setup</span>
<hr/>
<p>
This section explains how to properly format and configure
an extra hard drive for a NILM system. It is usually a good idea to place the Nilm Database on a seperate hard drive. This prevents the database from filling the primary drive to the point
where the system become unusable and also makes it easy to retrieve collected data from an installation by simply swapping out the extra harddrive.
</p>
<p><b>Formatting a drive</b> After installing the drive and booting the system, the first step to use
the drive is to place a usable filesystem on it. There are many tools that can be used for this but one of the easiest is GParted. This program must be run as root. From the command line type type the following:
</p>
<div class="alert alert-danger"><i class="icon icon-warning-sign"></i> Be very careful with gparted, formatting the primary drive will destroy the installation</div>
<section class="indented cm-view ">
<textarea class="cm-textarea-sh">
$ sudo gparted </textarea>
</section>
<p>
Select the extra drive from the dropdown menu as shown. Generally the extra drive should be <b>/dev/sdb</b>
but this is not always the case. If the drive already has multiple partitions this most likely means it is the primary drive.
</p>
<div class="row">
<div class="col-sm-12 col-md-6">
<div class="thumbnail">
<%=image_tag "hdd_gparted_1.png"%>
<div class="caption">
Carefully select the device node for the extra drive
</div>
</div>
</div>
<div class="col-sm-12 col-md-6">
<div class="thumbnail">
<%=image_tag "hdd_gparted_2.png"%>
<div class="caption">
Create a new msdos partition table. This will erase the drive.
</div>
</div>
</div>
</div>
<p>
Select <b>Device</b> > <b> Create Partition Table</b> to bring up the Create Partition dialog. Select <b>msdos</b> and click Apply. Select <b>Partition</b> > <b>New</b> to bring up the New Partition dialog. Add a new <b>ext4</b> partition to the drive and assign it the full extents of the disk (this is the default). Click Add to close the dialog. Finally click <b>Apply</b> to format the disk and then close the program.
<div class="row">
<div class="col-sm-12 col-md-6 col-md-offset-3">
<div class="thumbnail">
<%=image_tag "hdd_gparted_3.png"%>
<div class="caption">
Add an <b>ext4</b> partition to fill the disk
</div>
</div>
</div>
</div>
<p>
To use the drive for the Nilm database it must be mounted to the correct location in the filesystem. Edit <b>/etc/fstab</b> in a word processor and add the following line where /dev/sdX1 is the name of the drive you just formatted. The number one refers to the first (and only) partition. Note that you will need to run the word processor as root (sudo) in order to edit this file.
<pre>
/dev/sdX1 /opt/data ext4 errors=remount-ro 0 1 </pre>
When using an external drive as the primary storage volume, drive letters cannot be used since they change. The UUID is a unique drive partition identifier that should be used instead. Running <span class="text-mono">sudo lsblk /dev/sdX1</span> will print out the UUID. The new line in <b>/etc/fstab</b> should resemble the following.
<pre>
UUID=XXXXXXXX-XXXXX-XXXX-XXXX-XXXXXXXXXXXX /opt/data ext4 errors=remount-ro 0 1 </pre>
Run the following commands to mount the drive and setup the permissions. Use <b>df</b> to verify the configuration:
<section class="indented cm-view ">
<textarea class="cm-textarea-sh">
$ sudo service nilm-capture stop #stop data capture if it is already running
$ sudo mount -a
$ sudo chown -R nilm:nilm /opt/data #assign the drive to the nilm user
$ df -h</textarea>
</section>
</p>
<p>
The output from <b>df</b> should look similar to that below:
</p>
<pre>
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 1.8T 5.3G 1.7T 1% /
... other mount points ...
/dev/sdc1 917G 72M 871G 1% /opt/data</pre>
<p>If you have already configured your meters and want to start collecting data, run:</p>
<section class="indented cm-view ">
<textarea class="cm-textarea-sh">
$ sudo service nilm-capture start</textarea>
</section>
docs/index.rst
......@@ -3,121 +3,138 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
.. highlight:: bash
The Wattsworth Project
======================
.. raw:: html
.. NOTE::
<div class="alert alert-info">
See <a href="/joule">Joule</a> for information on the modular data processing framework
<br/>
See <a href="/web">Web</a> for information on the frontend user interface
</div>
* See `joule </joule>`_ for information on the backend interface (CLI)
* See `web </web>`_ for information on the frontend interface (user facing)
Active Plugins
--------------
The following plugins are active on this machine.
Click the name of a plugin to view the documentation.
Collect, process, and visualize IoT sensor data. Each Wattsworth is a complete
sensor network stack with support for:
.. raw:: html
1. **Data Collection** Collect and store highbandwidth data
2. **Data Processing** Build flexible signal processing pipelines.
3. **Data Visualization** View data at any resolution using an intuitive web interface
<table class="table">
<thead>
<tr><th>Name</th><th>Description</th></tr>
</thead>
<tbody id="plugins-table">
</tbody>
</table>
.. raw:: html
Nodes may be combined into a mesh or operate individually. There is no central
cloud server.
Install
+++++++
Installing the Software
-----------------------
On a fresh Ubuntu installation (>= 16.02 LTS) run the following to install
the Wattsworth software stack::
All of the software repositories are available at
https://git.wattsworth.net/wattsworth. The software has been tested on
64 bit Ubuntu Linux. While it is possible to run on Arm-based Single
Board Computers (eg Raspberry Pi), the software works best on x86 systems
such as the Intel NUC.
$> sudo apt-get update
$> sudo apt-get install puppet
$> git clone https://git.wattsworth.net/wattsworth/puppet.git
$> cd puppet
$> sudo puppet apply --modulepath=./modules --verbose site.pp
Use the Puppet repository to install the complete Wattsworth stack
Add a Reader Module
+++++++++++++++++++
.. code-block:: bash
Time series data is stored as streams. Streams are similar to files in a
filesystem. Copy the following to
``/etc/joule/stream_configs/random.conf`` to allocate a new stream.
$> sudo apt-get update
$> sudo apt-get install puppet
$> git clone https://git.wattsworth.net/wattsworth/puppet.git
$> cd puppet
$> sudo puppet apply --modulepath=./modules --verbose site.pp
.. code-block:: ini
System Configuration
--------------------
[Main]
name = Random Data
path = /demo/random
datatype = float32
keep = 1w
Make sure you set up data journaling on the nilmdb partition. This will prevent
data corruption if the computer looses power without properly shutting down.
[Element1]
name = rand1
Edit ``/etc/fstab`` and add the ``data=journal`` option to the parition with the
nilmdb database.
[Element2]
name = rand2
.. code-block:: bash
Reader modules read data from sensors and store it into a stream. Copy the
following to ``/etc/joule/module_configs/random_reader.conf`` to configure a
module to produce data for your new stream::
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
UUID=XXX / ext4 errors=remount-ro,data=journal 0 1
# add this --^
[Main]
exec_cmd = joule reader random 2 10
name = Demo Reader
If the partition is the root partition you must add this option directly to the volume
as well, otherwise the system will not boot properly. For example if your
root partition is on ``/dev/sda2``:
[Source]
# a reader has no inputs
.. code-block:: bash
[Destination]
output = /demo/random
$> sudo tune2fs -o journal_data /dev/sda2
From the terminal restart the Joule process to activate the module::
.. toctree::
:maxdepth: 2
:caption: Contents:
$> sudo service jouled restart
$> joule modules # show the active modules
View the Data
+++++++++++++
Open a browser and navigate to http://wattsworth.local. Log in as
**admin@wattsworth.local** with password **password**. From the main window
click the gear icon next to the local installation. This opens the
administration view (see ``admin help`` for more details). Click ``[Refresh]``
and the new data stream should appear in the file tree on the left. Navigate
back to the main page (click the header logo or use the browser back button) and
expand the file tree until you see the new stream. Click ``[Plot]`` and it
should display in the main window. See ``web help`` for information on plotting
data.
Add a Filter Module
+++++++++++++++++++
Filter modules process data generated by readers or by other filters. Filters
store their results into one or more output streams. Copy the
following to ``/etc/joule/stream_configs/averaged.conf`` to allocate
a new stream::
.. code-block:: ini
[Main]
name = Averaged Data
path = /demo/filtered
datatype = float32
keep = 1w
Indices and tables
==================
[Element1]
name = filtered1
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
[Element2]
name = filtered2
.. _web: /web
.. _joule: /joule
Configure a
.. raw:: html
From the terminal restart the Joule process to activate the filter::
<script type="text/javascript">
$> sudo service jouled restart
$> joule modules # show active modules
//hide the plugins section by default (in case plugins.txt is missing)
$("#active-plugins").hide();
View the Data
+++++++++++++
var plugins="";
var nonce=new Date().getTime(); //so we force the browser to get new data
From the web browser, go back to the administration view and refresh the
database. You should now see two streams. Back in the data view plot both
streams. The filtered stream should be the average of the random stream.
Contributing
------------
.. toctree::
:maxdepth: 2
:caption: Contents:
$.ajax({url: `_static/plugins.txt?nonce=${nonce}`, success:
function(result){
plugins = result.split('\n')
.reduce(function(acc,line){
if(line==""||line[0]=="#")
return acc;
acc.push(line.split(',').map(function(x){return x.trim()}));
return acc;
},[])
if(plugins.length>0)
$("#active-plugins").show();
plugins.map(function(plugin){
var link = plugin[2];
var name=`<a href="/${link}">${plugin[0]}</a>`;
var desc = plugin[1];
$("#plugins-table").append(`<tr><td>${name}</td><td>${desc}</td></tr>`)
})
}});
</script>
installation
configuration
plugins
docs/installation.rst 0 → 100644
Installation
============
All of the software repositories are available at
https://git.wattsworth.net/wattsworth. The software has been tested on
64 bit Ubuntu Linux. While it is possible to run on Arm-based Single
Board Computers (eg Raspberry Pi), the software works best on x86 systems
such as the Intel NUC.
Use the Puppet repository to install the complete Wattsworth stack
.. code-block:: bash
$> sudo apt-get update
$> sudo apt-get install puppet
$> git clone https://git.wattsworth.net/wattsworth/puppet.git
$> cd puppet
$> sudo puppet apply --modulepath=./modules --verbose site.pp
docs/plugins.rst 0 → 100644
Plugins
=======
.. raw:: html
<p id="no-plugins"> No plugins are currently installed </p>
<div id="has-plugins">
<p> Click a plugin name below to view the documentation </p>
<table class="table">
<thead>
<tr><th>Name</th><th>Description</th></tr>
</thead>
<tbody id="plugins-table">
</tbody>
</table>
</div>
.. raw:: html
<script type="text/javascript">
//hide the plugins section by default (in case plugins.txt is missing)
$("#has-plugins").hide();
var plugins="";
var nonce=new Date().getTime(); //so we force the browser to get new data
$.ajax({url: `_static/plugins.txt?nonce=${nonce}`, success:
function(result){
plugins = result.split('\n')
.reduce(function(acc,line){
if(line==""||line[0]=="#")
return acc;
acc.push(line.split(',').map(function(x){return x.trim()}));
return acc;
},[])
if(plugins.length>0){
$("#has-plugins").show();
$("#no-plugins").hide();
}
plugins.map(function(plugin){
var link = plugin[2];
var name=`<a href="/${link}">${plugin[0]}</a>`;
var desc = plugin[1];
$("#plugins-table").append(`<tr><td>${name}</td><td>${desc}</td></tr>`)
})
}});
</script>
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment