<dt>-u URL, --url URL</dt><dd> (default: http://localhost/nilmdb/) NilmDB server URL. Must be specified before the subcommand.</dd>
<dt>subcommand ...</dt><dd>The subcommand to run, followed by its arguments. This is required.</dd>
<dt>-h, --help</dt><dd>Print a help message with Usage information and details on all supported command-line arguments. This can also be specified after the subcom- mand, in which case the Usage and arguments of the subcommand are shown instead.</dd>
<dt>-v, --version</dt><dd>Print the nilmtool version.</dd>
</dl>
</div>
Environment Variables:
Some behaviors of nilmtool subcommands can be configured via environment variables.
.. raw:: html
<div class="block-indent">
<dl class="arglist">
<dt>NILMDB_URL</dt><dd> (default: http://localhost/nilmdb/) The default URL of the NilmDB server. This is used if --url is not specified, and can be set as an environment variable to avoid the need to specify it on each invocation of nilmtool.</dd>
<dt>TZ</dt><dd>(default: system default timezone) The timezone to use when parsing or displaying times. This is usually of the form America/New_York, using the standard TZ names from the IANA
Time Zone Database</dd>
</dl>
</div>
``nilmtool help``
+++++++++++++++++
Print more specific help for a subcommand. nilmtool help subcommand is the same as nilmtool subcommand ``--help``.
Usage::
nilmtool help [-h] subcommand
``nilmtool info``
+++++++++++++++++
Print server information such as software versions, database location, and disk space Usage.
Usage::
nilmtool info [-h]
Example
.. code-block:: bash
$> nilmtool info
Client version: 1.9.7
Server version: 1.9.7
Server URL: http://localhost/nilmdb/
Server database path: /home/nilmdb/db
Server disk space used by NilmDB: 143.87 GiB
Server disk space used by other: 378.93 GiB
Server disk space reserved: 6.86 GiB
Server disk space free: 147.17 GiB
``nilmtool create``
+++++++++++++++++++
Create a new empty stream at the specified path and with the specified layout.
Usage::
nilmtool create [-h] PATH LAYOUT
Arguments
.. raw:: html
<div class="block-indent">
<dl class="arglist">
<dt>PATH</dt><dd>Path of the new stream. Stream paths are similar to filesystem paths and must contain at least two components. For example, /foo/bar.</dd>
<dt>LAYOUT</dt><dd>Layout for the new stream. Layouts are of the form <type>_<count>. The <type> is one of those described in Section 2.2.3 of the <a href="#">NilmDB Reference Guide</a>, such as uint16, int64, or float32. <count> is a numeric count of how many data elements there are, per row. Streams store rows of homogeneous data only, and the largest supported <count> is 1024. Generally, counts should fall within a much lower range, typically between 1 and 32. For example, float32_8.</dd>
</dl>
</div>
``nilmtool rename``
+++++++++++++++++++
Rename or relocate a stream in the database from one path to another. Metadata and intervals, if any, are relocated to the new path name.
Usage::
nilmtool rename [-h] OLDPATH NEWPATH
Arguments
.. raw:: html
<div class="block-indent">
<dl class="arglist">
<dt>OLDPATH</dt><dd>Old existing stream path, e.g. /foo/old</dd>
<dt>NEWPATH</dt><dd>New stream path, e.g. /foo/bar/new</dd>
</dl>
</div>
Notes
Metadata contents are not changed by this operation. Any software tools that
store and use path names stored in metadata keys or values will need to update
them accordingly.
``nilmtool list``
+++++++++++++++++
List streams available in the database, optionally filtering by path, and
optionally including extended stream info and intervals.
<dt>PATH</dt><dd>(default: *) If paths are specified, only streams that
match the given paths are shown. Wildcards are accepted; for example,
/sharon/* will list all streams with a path beginning with /sharon/.
Note that, to prevent wildcards from being interpreted by the shell,
they should be quoted at the command line; for example:
<pre>
$> nilmtool list "/sharon/*"
$> nilmtool list "*raw"</textarea></pre>
</dd>
<dt>-E, --ext</dt><dd>Show extended stream information, like interval extents, total rows of data present, and total amount of time covered by the stream’s intervals.</dd>
<dt>-T, --timestamp-raw</dt><dd>When displaying timestamps in the output, show raw timestamp values from the NilmDB database rather than converting to human-readable times. Raw values are typically measured in microseconds since the Unix time epoch (1970/01/01 00:00 UTC).</dd>
<dt>-l, --layout</dt><dd>Display the stream layout next to the path name.</dd>
<dt>-n, --no-decim</dt><dd>Omit streams with paths containing the string ``~decim-``, to avoid cluttering the output with decimated streams.</dd>
<dt>-d, --detail</dt><dd>In addition to the normal output, show the time intervals present in each stream. See also nilmtool intervals in Section 3.2.3.7 of the <a href="#">NilmDB Reference Guide</a>, which can display more details about the intervals.</dd>
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp for intervals (free-form, inclusive).</dd>
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp for intervals (free-form, noninclusive).</dd>
</dl>
</div>
``nilmtool intervals``
++++++++++++++++++++++
List intervals in a stream, similar to ``nilmtool list --detail``, but with
options for calculating set-differences between intervals of two streams, and
for optimizing the output by joining adjacent intervals.
<dt>PATH</dt><dd>List intervals for this path.</dd>
<dt>-d DIFFPATH, --diff DIFFPATH</dt><dd>(default: none) If specified, perform a set-difference by subtract the intervals in this path; that is, only show interval ranges that are present in the original path but not present in diffpath.</dd>
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp for intervals (free-form, inclusive).</dd>
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp for intervals (free-form, noninclusive).</dd>
<dt>-T, --timestamp-raw</dt><dd>(default: min) (default: max) When displaying timestamps in the output, show raw timestamp values from the NilmDB database rather than converting to human-readable times. Raw values are typically measured in microseconds since the Unix time epoch (1970/01/01 00:00 UTC).</dd>
<dt>-o, --optimize</dt><dd>Optimize the interval output by merging adjacent intervals. For example, the two intervals [1 → 2⟩ and [2 → 5⟩ would be displayed as one interval [1 → 5⟩.</dd>
</dl>
</div>
``nilmtool metadata``
+++++++++++++++++++++
Get, set, update, or delete the key/value metadata associated with a stream.
<dt>PATH</dt><dd>Path of the stream for which to manage metadata. Required, and must be specified before the action arguments.</dd>
</dl>
</div>
Action Arguments: These actions are mutually exclusive.
.. raw:: html
<div class="block-indent">
<dl class="arglist">
<dt>-g [KEY ...], --get [KEY ...]</dt><dd>(default: all) Get and print metadata for the specified key(s). If none are specified, print metadata for all keys. Keys are printed as key=value, one per line.</dd>
<dt>-s [KEY=VALUE ...], --set [KEY=VALUE ...]</dt><dd>Set metadata. Keys and values are specified as a key=value string. This replaces all existing metadata on the stream with the provided keys; any keys present in the database but not specified on the command line are removed.</dd>
<dt>-u [KEY=VALUE ...], --update [KEY=VALUE ...]</dt><dd>Update metadata. Keys and values are specified as a key=value string. This is similar to --set, but only adds or changes metadata keys; keys that are present in the database but not specified on the command line are left unchanged.</dd>
<dt>-d [KEY ...], --delete [KEY ...]</dt><dd>(default: all) Delete metadata for the specified key(s). If none are specified, delete all metadata for the stream. </dd>
<dt>PATH</dt><dd>Path of the stream into which to insert data. The format of the input data must match the layout of the stream.</dd>
<dt>FILE</dt><dd>(default: standard input) Input data filename, which must be formatted as uncompressed plain text. Default is to read the input from stdin.</dd>
<i>Timestamping</i>: To add timestamps to data that does not already have it, specify both of these arguments. The added timestamps are based on the interval start time and the given data rate.
<div class="block-indent">
<dl class="arglist">
<dt>-t, --timestamp</dt><dd>Add timestamps to each line</dd>
<dt>-r RATE, --rate RATE</dt><dd> Data rate, in Hz</dd>
</dl>
</div>
<i>Start Time</i>: The start time may be manually specified, or it can be determined from the input filename, based on the following options.
<div class="block-indent">
<dl class="arglist">
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp for the new interval (free-form, inclusive)</dd>
<dt>-f, --filename</dt><dd>Use filename to determine start time</dd>
</dl>
</div>
<i>End Time</i>: The ending time should be manually specified. If timestamps are being added, this can be omitted, in which case the end of the interval is set to the last timestamp plus one microsecond.
<div class="block-indent">
<dl class="arglist">
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp for the new interval (free-form, noninclusive)</dd>
</dl>
</div>
``nilmtool extract``
++++++++++++++++++++
Extract rows of data from a specified time interval in a stream, or output a
count of how many rows are present in the interval.
Usage::
nilmtool extract [-h] -s TIME -e TIME [-B] [-b] [-a] [-m] [-T] [-c]
path
Arguments
.. raw:: html
<div class="block-indent">
<dl class="arglist">
<dt>PATH</dt><dd>Path of the stream from which to extract data.</dd>
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp to extract (free-form, inclusive)</dd>
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp to extract (free-form, noninclusive)</dd>
</dl>
</div>
<i>Output Formatting</i>
<div class="block-indent">
<dl class="arglist">
<dt>-B, --binary</dt><dd>Output raw binary data instead of the usual text format. For details on the text and binary formatting, see the documentation of HTTP call /stream/insert in Section 3.2.1.13.</dd>
<dt>-b, --bare</dt><dd>Omit timestamps from each line of the output.</dd>
<dt>-a, --annotate</dt><dd>Include comments at the beginning of the output with information about the stream. Comments are lines beginning with #.</dd>
<dt>-m, --markup</dt><dd>Include comments in the output with information that denotes where the stream’s internal intervals begin and end. See the documentation of the markup parameter to HTTP call /stream/extract in Section 3.2.1.14 for details on the format of the comments.</dd>
<dt>-T, --timestamp-raw</dt><dd>Use raw integer timestamps in the --annotate output instead of human- readable strings.</dd>
<dt>-c, --count</dt><dd>Instead of outputting the data, output a count of how many rows are present in the given time interval. This is fast as it does not transfer the data from the server.</dd>
</dl>
</div>
``nilmtool remove``
+++++++++++++++++++
Remove all data from a specified time range within the stream at /PATH/.
Multiple streams may be specified, and wildcards are supported; the same time
range will be removed from all matching streams.
Usage::
nilmtool remove [-h] -s TIME -e TIME [-q] [-c] path [path ...]
Arguments
.. raw:: html
<div class="block-indent">
<dl class="arglist">
<dt>PATH</dt><dd> Path(s) of streams. Wildcards are supported. At least one path must provided.</dd>
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp of data to remove (free-form, inclusive, required).</dd>
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp of data to remove (free-form, noninclusive, required).</dd>
</dl>
</div>
<i>Output Format</i>
<div class="block-indent">
<dl class="arglist">
<dt>-q, --quiet</dt><dd>By default, matching path names are printed when removing from multiple paths. With this option, path names are not printed.</dd>
<dt>-c, --count</dt><dd>Display a count of the number of rows of data that were removed from each path.</dd>
<dt>PATH</dt><dd>Path(s) of streams. Wildcards are supported. At least one path must provided.</dd>
<dt>-R, --remove</dt><dd>If specified, all data is removed before destroying the stream. Equivalent to first running <span class="mono">nilmtool remove -s min -e max path``.</dd>
<dt>-q, --quiet</dt><dd>Don’t display names when destroying multiple paths</dd>
</dl>
</div>
``nilm-copy``
-------------
Copy data and metadata from one stream to another. The source and destination
streams can reside on different servers. Both streams must have the same layout.
Only regions of time that are present in the source, and not yet present in the
destination, are processed. This program can therefore be re-run with the same
command-line arguments multiple times, and it will only process the newly
<dt>-u URL, --url URL</dt><dd> (default: http://localhost/nilmdb/) NilmDB server URL for the source stream.</dd>
<dt>-U DESTURL, --dest-url DESTURL</dt><dd> (default: same as URL) NilmDB server URL for the destination stream. If unspecified, the same URL is used for both source and destination.</dd>
<dt>-D, --dry-run</dt><dd>Just print intervals that would be processed, and exit.</dd>
<dt>-F, --force-metadata</dt><dd>Metadata is copied from the source to the destination. By default, an error is returned if the destination stream metadata conflicts with the source stream metadata. Specify this flag to always overwrite the destination values with those from the source stream.</dd>
<dt>-n, --nometa</dt><dd>Don’t copy or check metadata at all.</dd>
<dt>-s TIME, --start TIME</dt><dd>(default: min) Starting timestamp of data to copy (free-form, inclusive).</dd>
<dt>-e TIME, --end TIME</dt><dd>(default: max) Ending timestamp of data to copy (free-form, noninclusive).</dd>
<dt>SRCPATH</dt><dd>Path of the source stream (on the source server).</dd>
<dt>DESTPATH</dt><dd>Path of the destination stream (on the destination server).</dd>
</dl>
</div>
``nilm-copy-wildcard``
----------------------
Copy data and metadata, from multiple streams, between two servers. Similar to nilm-copy, except:
* Wildcards and multiple paths are supported in the stream names.
* Streams must always be copied between two servers.
* Stream paths must match on the source and destination server.
* If a stream does not exist on the destination server, it is created with the correct layout automatically.
<dt>-u URL, --url URL</dt><dd>(default: http://localhost/nilmdb/) NilmDB server URL for the source stream.</dd>
<dt>-U DESTURL, --dest-url DESTURL</dt><dd>(default: same as URL) NilmDB server URL for the destination stream. If unspecified, the same URL is used for both source and destination.</dd>
<dt>-D, --dry-run</dt><dd>Just print intervals that would be processed, and exit.</dd>
<dt>-F, --force-metadata</dt><dd>Overwrite destination metadata even if it conflicts with the values in the “metadata” section below.</dd>
<dt>-s TIME, --start TIME</dt><dd>(default: min) Starting timestamp of data to decimate (free-form, inclusive).</dd>
<dt>-e TIME, --end TIME</dt><dd>(default: max) Ending timestamp of data to decimate (free-form, noninclusive).</dd>
<dt>-f FACTOR, --factor FACTOR</dt><dd>(default: 4) Set the decimation factor. For a source stream with n rows, the output stream will have n/FACTOR rows.</dd>
<dt>SRCPATH</dt><dd>Path of the source stream (on the source server).</dd>
<dt>DESTPATH</dt><dd>Path of the destination stream (on the destination server).</dd>
</dl>
</div>
The destination stream has the following metadata keys added:
decimate_source
The source stream from which this data was decimated.
decimate_factor
The decimation factor used.
``nilm-decimate-auto``
----------------------
Automatically create multiple decimation levels using from a single source
stream, continuing until the last decimated level contains fewer than 500 rows
total. Decimations are performed using nilm-decimate (Section 3.4.2.4).
Wildcards and multiple paths are accepted. Destination streams are automatically
named based on the source stream name and the total decimation factor; for
example, ``/test/raw~decim-4``, ``/test/raw~decim-16``, etc. Streams containing
the string "``~decim-``" are ignored when matching wildcards.
<dt>-u URL, --url URL</dt><dd> (default: http://localhost/nilmdb/) NilmDB server URL for the source and destination streams.</dd>
<dt>-F, --force-metadata</dt><dd>Overwrite destination metadata even if it conflicts with the values in the “metadata” section above.</dd>
<dt>-f FACTOR, --factor FACTOR</dt><dd>(default: 4) Set the decimation factor. Each decimation level will have 1/FACTOR as many rows as the previous level.</dd>
<dt>PATH [...]</dt><dd>One or more paths to decimate. Wildcards are accepted.</dd>
</dl>
</div>
``nilm-insert``
---------------
Insert a large amount of text-formatted data from an external source like
ethstream. This is a higher-level tool than nilmtool insert in that it attempts
to intelligently manage timestamps. The general concept is that it tracks two
timestamps:
1. The data timestamp is the precise timestamp corresponding to a particular row of data, and is the timestamp that gets inserted into the database. It increases by data_delta for every row of input. data_delta can come from one of two sources. If --delta is specified, it is pulled from the first column of data. If --rate is specified, data_delta is set to a fixed value of 1/RATE.
2. The clock timestamp is the less precise timestamp that gives the absolute time. It can come from two sources. If --live is specified, it is pulled directly from the system clock. If --file is specified, it is extracted from the input file every time a new file is opened for read, and from comments that appear in the files.
Small discrepancies between data and clock are ignored. If the data timestamp ever differs from the clock timestamp by more than max_gap seconds:
* If data is running behind, there is a gap in the data, so the timestamp is stepped forward to match clock.
* If data is running ahead, there is overlap in the data, and an error is returned. If --skip is specified, then instead of returning an error, data is dropped and the remainder of the current file is skipped.
<dt>-u URL, --url URL</dt><dd> (default: http://localhost/nilmdb/) NilmDB server URL.</dd>
<dt>-D, --dry-run</dt><dd>Parse files and print information, but don’t insert any data. Useful for verification before making changes to the database.</dd>
<dt>-s, --skip</dt><dd>Skip the remainder of input files if the data timestamp runs too far ahead of the clock timestamp. Useful when inserting a large directory of existing files with inaccurate timestamps.</dd>
<dt>-m SEC, --max-gap SEC</dt><dd>(default: 10.0) Maximum discrepancy between the clock and data timestamps.</dd>
</dl>
</div>
<i>Data timestamp</i>
<div class="block-indent">
<dl class="arglist">
<dt>-r RATE, --rate RATE</dt><dd>(default: 8000.0) data_delta is constant 1/RATE (in Hz).</dd>
<dt>-d, --delta</dt><dd>data_delta is provided as the first number on each input line.</dd>
</dl>
</div>
<i>Clock timestamp</i>
<div class="block-indent">
<dl class="arglist">
<dt>-l, --live</dt><dd>Use the live system time for the clock timestamp. This is most useful when piping in data live from a capture device.</dd>
<dt>-f, --file</dt><dd>Use filename and file comments for the clock timestamp. This is most useful when reading previously saved data.</dd>
<dt>-o SEC, --offset-filename SEC</dt><dd>(default: −3600.0) Offset to add to timestamps in filenames, when using --file. The default accounts for the existing practice of naming capture files based on the end of the hour in which they were recorded. The filename timestamp plus this offset should equal the time that the first row of data in the file was captured.</dd>
<dt>-O SEC, --offset-comment SEC</dt><dd>(default: 0.0) Offset to add to timestamps in comments, when using --file. The comment timestamp plus this offset should equal the time that the next row of data was captured.</dd>
</dl>
</div>
<i>Path and Input</i>
<div class="block-indent">
<dl class="arglist">
<dt>PATH</dt><dd>Path of the stream into which to insert data. The layout of the path must match the input data.</dd>
<dt>INFILE [...]</dt><dd>(default: standard input) Input data filename(s). Filenames ending with .gz are transparently decompressed as they are read. The default is to read the input from stdin.</dd>
</dl>
</div>
.. DANGER::
The following tools provide low level access to the NILM and are not
required for normal system use. Be careful running them as they may
corrupt the database or cause loss of data.
``nilmdb-server``
-----------------
Run a standalone NilmDB server. Note that the NilmDB server is typically run
as a WSGI process managed by Apache. This program runs NilmDB
<dt>-v, --version</dt><dd> Print the installed NilmDB version.</dd>
<dt>-a ADDRESS, --address ADDRESS</dt><dd> (default: 0.0.0.0) Only listen on the given IP address. The default is to listen on all addresses.</dd>
<dt>-p PORT, --port PORT</dt><dd>(default: 12380) Listen on the given TCP port.</dd>
<dt>-d DATABASE, --database DATABASE</dt><dd>(default: ./db) Local filesystem directory of the NilmDB database.</dd>
<dt>-q, --quiet</dt><dd>Silence output.</dd>
</dl>
</div>
<i>Debug Options</i>
<div class="block-indent">
<dl class="arglist">
<dt>-t, --traceback</dt><dd>Provide tracebacks in the error response for client errors (HTTP status codes 400 - 499). Normally, tracebacks are only provided for server errors (HTTP status codes 500 - 599).</dd>
<dt>-y, --yappi</dt><dd>Run under the yappi profiler and invoke an interactive shell afterwards. Not intended for normal operation.</dd>
</dl>
</div>
``nilmdb-fsck``
---------------
Check database consistency, and optionally repair errors automatically, when
possible. Running this may be necessary after an improper shutdown or other
corruption has occurred. This program will refuse to run if the database is
currently locked by any other process, like the Apache webserver; such programs
should be stopped first. This is run automatically on system boot for the Joule
database.
Usage::
nilmdb-fsck [-h] [-v] [-f] [-n] database
Arguments
.. raw:: html
<div class="block-indent">
<dl class="arglist">
<dt>DATABASE</dt><dd>Local filesystem directory of the NilmDB database to check.</dd>
<dt>-f, --fix</dt><dd>Attempt to fix errors when possible. Note that this may involve removing intervals or data.</dd>
<dt>-n, --no-data</dt><dd>Skip the slow full-data check. The earlier, faster checks are likely to find most database corruption, so the data checks may be unnecessary.</dd>
<dt>-h, --help</dt><dd>Print a help message with Usage information and details.</dd>
<dt>-v, --version</dt><dd>Print the installed NilmDB version. Generally, you should ensure that the version of nilmdb-fsck is newer than the NilmDB version that created, or last used, the given database.</dd>
</dl>
</div>
``nilm-cleanup``
----------------
Clean up old data from streams, using a configuration file to specify which data
to remove. The configuration file is a text file in the following format::
[/stream/path]
keep = 3w # keep up to 3 weeks of data
rate = 8000 # optional, used for the --estimate option
decimated = false # whether to delete decimated data too
[*/wildcard/path]
keep = 3.5m # or 2520h or 105d or 15w or 0.29y
Stream paths are specified inside square brackets (``[]``) and are followed by configuration
keywords for the matching streams. Paths can contain wildcards. Supported keywords are:
``keep``
How much data to keep. Supported suffixes are h for hours, d for days, w for weeks, m for months, and y for years.
``rate``
(default: automatic) Expected data rate. Only used by the ``--estimate option``. If not specified, the rate is guessed based on the existing data in the stream.
``decimated``
(default: true) If true, delete decimated data too. For stream path /A/B, this includes any stream matching the wildcard /A/B~decim*. If specified as false, no special treatment is applied to such streams.
The value keep is a maximum amount of data, not a cutoff time. When cleaning
data, the oldest data in the stream will be removed, until the total remaining
amount of data is less than or equal to keep. This means that data older than
keep will remain if insufficient newer data is present; for example, if new data
ceases to be inserted, old data will cease to be deleted.
<dt>-u URL, --url URL</dt><dd> (default: http://localhost/nilmdb/) NilmDB server URL.</dd>
<dt>-y, --yes</dt><dd>Actually remove the data. By default, nilm-cleanup only prints what it would have removed, but leaves the data intact.</dd>
<dt>-e, --estimate</dt><dd>Instead of removing data, print an estimated report of the maximum amount of disk space that will be used by the cleaned-up streams. This uses the on-disk size of the stream layout, the estimated data rate, and the space required by decimation levels. Streams not matched in the con- figuration file are not included in the total.</dd>
<dt>CONFIGFILE</dt><dd> Path to the configuration file. </dd>
<h1>Database Reference<aclass="headerlink"href="#database-reference"title="Permalink to this headline">¶</a></h1>
<p>Joule uses a NilmDB database instance to store data streams. NilmDB was developed
at MIT by Dr. James Paris. Full documentation is available in his thesis available
at <aclass="reference external"href="https://dspace.mit.edu/handle/1721.1/84720">MIT DSpace</a>. This documentation
covers the command line interface for interacting the the Joule NilmDB instance.</p>
<p>Command-line arguments can often be supplied in both short and long
forms, and many arguments are optional. The following documentation uses these
conventions:</p>
<ulclass="simple">
<li>An argument that takes an additional parameter is denoted <codeclass="docutils literal"><spanclass="pre">-f</span><spanclass="pre">FILE</span></code>.</li>
<li>The syntax <codeclass="docutils literal"><spanclass="pre">-f</span><spanclass="pre">FILE,</span><spanclass="pre">--file</span><spanclass="pre">FILE</span></code> indicates that either the short form (-f) or long form (–file) can be used interchangeably.</li>
<li>Pipes (<codeclass="docutils literal"><spanclass="pre">A|B</span></code>) indicate that either <codeclass="docutils literal"><spanclass="pre">A</span></code> or <codeclass="docutils literal"><spanclass="pre">B</span></code> can be specified, but not both.</li>
<li>Curly braces (<codeclass="docutils literal"><spanclass="pre">{}</span></code>) indicate a list of mutually-exclusive argument choices.</li>
</ul>
<p>Many of the programs support arguments that represent a NilmDB timestamp. This
timestamp is specified as a free-form string, as supported by the <strong>parse_time</strong>
client library function, described in Section 3.2.2.4 of the NilmDB reference
guide. Examples of accepted formats are shown in Table 3-19 on page 133 of that
document.</p>
<divclass="section"id="nilmtool">
<h2><codeclass="docutils literal"><spanclass="pre">nilmtool</span></code><aclass="headerlink"href="#nilmtool"title="Permalink to this headline">¶</a></h2>
<p>Tools for interacting with the database are wrapped in <codeclass="docutils literal"><spanclass="pre">nilmtool</span></code>, a
monolithic multi-purpose program that provides command-line access to most of
the NilmDB functionality. Global operation is described first followed by
specific documentation for each subcommand.</p>
<p>The command-line syntax provides the ability to execute sub- commands: first,
global arguments that affect the behavior of all subcommands can be specified,
followed by one subcommand name, followed by arguments for that subcommand. Each
defines its own arguments and is documented below.</p>
<dt>-u URL, --url URL</dt><dd> (default: http://localhost/nilmdb/) NilmDB server URL. Must be specified before the subcommand.</dd>
<dt>subcommand ...</dt><dd>The subcommand to run, followed by its arguments. This is required.</dd>
<dt>-h, --help</dt><dd>Print a help message with Usage information and details on all supported command-line arguments. This can also be specified after the subcom- mand, in which case the Usage and arguments of the subcommand are shown instead.</dd>
<dt>-v, --version</dt><dd>Print the nilmtool version.</dd>
</dl>
</div><p>Environment Variables:</p>
<p>Some behaviors of nilmtool subcommands can be configured via environment variables.</p>
<divclass="block-indent">
<dlclass="arglist">
<dt>NILMDB_URL</dt><dd> (default: http://localhost/nilmdb/) The default URL of the NilmDB server. This is used if --url is not specified, and can be set as an environment variable to avoid the need to specify it on each invocation of nilmtool.</dd>
<dt>TZ</dt><dd>(default: system default timezone) The timezone to use when parsing or displaying times. This is usually of the form America/New_York, using the standard TZ names from the IANA
Time Zone Database</dd>
</dl>
</div><divclass="section"id="nilmtool-help">
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">help</span></code><aclass="headerlink"href="#nilmtool-help"title="Permalink to this headline">¶</a></h3>
<p>Print more specific help for a subcommand. nilmtool help subcommand is the same as nilmtool subcommand <codeclass="docutils literal"><spanclass="pre">--help</span></code>.</p>
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">info</span></code><aclass="headerlink"href="#nilmtool-info"title="Permalink to this headline">¶</a></h3>
<p>Print server information such as software versions, database location, and disk space Usage.</p>
<divclass="highlight-bash"><divclass="highlight"><pre><span></span>$> nilmtool info
Client version: <spanclass="m">1</span>.9.7
Server version: <spanclass="m">1</span>.9.7
Server URL: http://localhost/nilmdb/
Server database path: /home/nilmdb/db
Server disk space used by NilmDB: <spanclass="m">143</span>.87 GiB
Server disk space used by other: <spanclass="m">378</span>.93 GiB
Server disk space reserved: <spanclass="m">6</span>.86 GiB
Server disk space free: <spanclass="m">147</span>.17 GiB
</pre></div>
</div>
</div>
<divclass="section"id="nilmtool-create">
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">create</span></code><aclass="headerlink"href="#nilmtool-create"title="Permalink to this headline">¶</a></h3>
<p>Create a new empty stream at the specified path and with the specified layout.</p>
<dt>PATH</dt><dd>Path of the new stream. Stream paths are similar to filesystem paths and must contain at least two components. For example, /foo/bar.</dd>
<dt>LAYOUT</dt><dd>Layout for the new stream. Layouts are of the form <type>_<count>. The <type> is one of those described in Section 2.2.3 of the <ahref="#">NilmDB Reference Guide</a>, such as uint16, int64, or float32. <count> is a numeric count of how many data elements there are, per row. Streams store rows of homogeneous data only, and the largest supported <count> is 1024. Generally, counts should fall within a much lower range, typically between 1 and 32. For example, float32_8.</dd>
</dl>
</div></div>
<divclass="section"id="nilmtool-rename">
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">rename</span></code><aclass="headerlink"href="#nilmtool-rename"title="Permalink to this headline">¶</a></h3>
<p>Rename or relocate a stream in the database from one path to another. Metadata and intervals, if any, are relocated to the new path name.</p>
<dt>OLDPATH</dt><dd>Old existing stream path, e.g. /foo/old</dd>
<dt>NEWPATH</dt><dd>New stream path, e.g. /foo/bar/new</dd>
</dl>
</div><p>Notes</p>
<blockquote>
<div>Metadata contents are not changed by this operation. Any software tools that
store and use path names stored in metadata keys or values will need to update
them accordingly.</div></blockquote>
</div>
<divclass="section"id="nilmtool-list">
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">list</span></code><aclass="headerlink"href="#nilmtool-list"title="Permalink to this headline">¶</a></h3>
<p>List streams available in the database, optionally filtering by path, and
optionally including extended stream info and intervals.</p>
<dt>PATH</dt><dd>(default: *) If paths are specified, only streams that
match the given paths are shown. Wildcards are accepted; for example,
/sharon/* will list all streams with a path beginning with /sharon/.
Note that, to prevent wildcards from being interpreted by the shell,
they should be quoted at the command line; for example:
<pre>
$> nilmtool list "/sharon/*"
$> nilmtool list "*raw"</textarea></pre>
</dd>
<dt>-E, --ext</dt><dd>Show extended stream information, like interval extents, total rows of data present, and total amount of time covered by the stream’s intervals.</dd>
<dt>-T, --timestamp-raw</dt><dd>When displaying timestamps in the output, show raw timestamp values from the NilmDB database rather than converting to human-readable times. Raw values are typically measured in microseconds since the Unix time epoch (1970/01/01 00:00 UTC).</dd>
<dt>-l, --layout</dt><dd>Display the stream layout next to the path name.</dd>
<dt>-n, --no-decim</dt><dd>Omit streams with paths containing the string ``~decim-``, to avoid cluttering the output with decimated streams.</dd>
<dt>-d, --detail</dt><dd>In addition to the normal output, show the time intervals present in each stream. See also nilmtool intervals in Section 3.2.3.7 of the <ahref="#">NilmDB Reference Guide</a>, which can display more details about the intervals.</dd>
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp for intervals (free-form, inclusive).</dd>
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp for intervals (free-form, noninclusive).</dd>
</dl>
</div></div>
<divclass="section"id="nilmtool-intervals">
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">intervals</span></code><aclass="headerlink"href="#nilmtool-intervals"title="Permalink to this headline">¶</a></h3>
<p>List intervals in a stream, similar to <codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">list</span><spanclass="pre">--detail</span></code>, but with
options for calculating set-differences between intervals of two streams, and
for optimizing the output by joining adjacent intervals.</p>
<dt>PATH</dt><dd>List intervals for this path.</dd>
<dt>-d DIFFPATH, --diff DIFFPATH</dt><dd>(default: none) If specified, perform a set-difference by subtract the intervals in this path; that is, only show interval ranges that are present in the original path but not present in diffpath.</dd>
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp for intervals (free-form, inclusive).</dd>
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp for intervals (free-form, noninclusive).</dd>
<dt>-T, --timestamp-raw</dt><dd>(default: min) (default: max) When displaying timestamps in the output, show raw timestamp values from the NilmDB database rather than converting to human-readable times. Raw values are typically measured in microseconds since the Unix time epoch (1970/01/01 00:00 UTC).</dd>
<dt>-o, --optimize</dt><dd>Optimize the interval output by merging adjacent intervals. For example, the two intervals [1 → 2⟩ and [2 → 5⟩ would be displayed as one interval [1 → 5⟩.</dd>
</dl>
</div></div>
<divclass="section"id="nilmtool-metadata">
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">metadata</span></code><aclass="headerlink"href="#nilmtool-metadata"title="Permalink to this headline">¶</a></h3>
<p>Get, set, update, or delete the key/value metadata associated with a stream.</p>
<dt>PATH</dt><dd>Path of the stream for which to manage metadata. Required, and must be specified before the action arguments.</dd>
</dl>
</div><p>Action Arguments: These actions are mutually exclusive.</p>
<divclass="block-indent">
<dlclass="arglist">
<dt>-g [KEY ...], --get [KEY ...]</dt><dd>(default: all) Get and print metadata for the specified key(s). If none are specified, print metadata for all keys. Keys are printed as key=value, one per line.</dd>
<dt>-s [KEY=VALUE ...], --set [KEY=VALUE ...]</dt><dd>Set metadata. Keys and values are specified as a key=value string. This replaces all existing metadata on the stream with the provided keys; any keys present in the database but not specified on the command line are removed.</dd>
<dt>-u [KEY=VALUE ...], --update [KEY=VALUE ...]</dt><dd>Update metadata. Keys and values are specified as a key=value string. This is similar to --set, but only adds or changes metadata keys; keys that are present in the database but not specified on the command line are left unchanged.</dd>
<dt>-d [KEY ...], --delete [KEY ...]</dt><dd>(default: all) Delete metadata for the specified key(s). If none are specified, delete all metadata for the stream. </dd>
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">insert</span></code><aclass="headerlink"href="#nilmtool-insert"title="Permalink to this headline">¶</a></h3>
<p>Insert data into a stream. This is a relatively low-level interface analogous to
the /stream/insert HTTP interface described in Section 3.2.1.13 on the <a
href=”#”>NilmDB Reference Guide</a>. This is the program that should be used
when a fixed quantity of text-based data is being inserted into a single
interval, with a known start and end time. If the input data does not already
have timestamps, they can be optionally added based on the start time and a
known data rate. In many cases, using the separate <codeclass="docutils literal"><spanclass="pre">nilm-insert</span></code> program is
preferable, particularly when dealing with large amounts of pre-recorded data,
<dt>PATH</dt><dd>Path of the stream into which to insert data. The format of the input data must match the layout of the stream.</dd>
<dt>FILE</dt><dd>(default: standard input) Input data filename, which must be formatted as uncompressed plain text. Default is to read the input from stdin.</dd>
<i>Timestamping</i>: To add timestamps to data that does not already have it, specify both of these arguments. The added timestamps are based on the interval start time and the given data rate.
<divclass="block-indent">
<dlclass="arglist">
<dt>-t, --timestamp</dt><dd>Add timestamps to each line</dd>
<dt>-r RATE, --rate RATE</dt><dd> Data rate, in Hz</dd>
</dl>
</div>
<i>Start Time</i>: The start time may be manually specified, or it can be determined from the input filename, based on the following options.
<divclass="block-indent">
<dlclass="arglist">
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp for the new interval (free-form, inclusive)</dd>
<dt>-f, --filename</dt><dd>Use filename to determine start time</dd>
</dl>
</div>
<i>End Time</i>: The ending time should be manually specified. If timestamps are being added, this can be omitted, in which case the end of the interval is set to the last timestamp plus one microsecond.
<divclass="block-indent">
<dlclass="arglist">
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp for the new interval (free-form, noninclusive)</dd>
</dl>
</div></div>
<divclass="section"id="nilmtool-extract">
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">extract</span></code><aclass="headerlink"href="#nilmtool-extract"title="Permalink to this headline">¶</a></h3>
<p>Extract rows of data from a specified time interval in a stream, or output a
count of how many rows are present in the interval.</p>
<dt>PATH</dt><dd>Path of the stream from which to extract data.</dd>
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp to extract (free-form, inclusive)</dd>
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp to extract (free-form, noninclusive)</dd>
</dl>
</div>
<i>Output Formatting</i>
<divclass="block-indent">
<dlclass="arglist">
<dt>-B, --binary</dt><dd>Output raw binary data instead of the usual text format. For details on the text and binary formatting, see the documentation of HTTP call /stream/insert in Section 3.2.1.13.</dd>
<dt>-b, --bare</dt><dd>Omit timestamps from each line of the output.</dd>
<dt>-a, --annotate</dt><dd>Include comments at the beginning of the output with information about the stream. Comments are lines beginning with #.</dd>
<dt>-m, --markup</dt><dd>Include comments in the output with information that denotes where the stream’s internal intervals begin and end. See the documentation of the markup parameter to HTTP call /stream/extract in Section 3.2.1.14 for details on the format of the comments.</dd>
<dt>-T, --timestamp-raw</dt><dd>Use raw integer timestamps in the --annotate output instead of human- readable strings.</dd>
<dt>-c, --count</dt><dd>Instead of outputting the data, output a count of how many rows are present in the given time interval. This is fast as it does not transfer the data from the server.</dd>
</dl>
</div></div>
<divclass="section"id="nilmtool-remove">
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">remove</span></code><aclass="headerlink"href="#nilmtool-remove"title="Permalink to this headline">¶</a></h3>
<p>Remove all data from a specified time range within the stream at /PATH/.
Multiple streams may be specified, and wildcards are supported; the same time
range will be removed from all matching streams.</p>
<dt>PATH</dt><dd> Path(s) of streams. Wildcards are supported. At least one path must provided.</dd>
<dt>-s TIME, --start TIME</dt><dd>Starting timestamp of data to remove (free-form, inclusive, required).</dd>
<dt>-e TIME, --end TIME</dt><dd>Ending timestamp of data to remove (free-form, noninclusive, required).</dd>
</dl>
</div>
<i>Output Format</i>
<divclass="block-indent">
<dlclass="arglist">
<dt>-q, --quiet</dt><dd>By default, matching path names are printed when removing from multiple paths. With this option, path names are not printed.</dd>
<dt>-c, --count</dt><dd>Display a count of the number of rows of data that were removed from each path.</dd>
<h3><codeclass="docutils literal"><spanclass="pre">nilmtool</span><spanclass="pre">destroy</span></code><aclass="headerlink"href="#nilmtool-destroy"title="Permalink to this headline">¶</a></h3>
<p>Destroy the stream at the specified path(s); the opposite of nilmtool create.
Metadata related to the stream is permanently deleted. All data must be removed
before a stream can be destroyed. Wildcards are supported.</p>
<h2><codeclass="docutils literal"><spanclass="pre">nilm-copy</span></code><aclass="headerlink"href="#nilm-copy"title="Permalink to this headline">¶</a></h2>
<p>Copy data and metadata from one stream to another. The source and destination
streams can reside on different servers. Both streams must have the same layout.
Only regions of time that are present in the source, and not yet present in the
destination, are processed. This program can therefore be re-run with the same
command-line arguments multiple times, and it will only process the newly
<h2><codeclass="docutils literal"><spanclass="pre">nilm-copy-wildcard</span></code><aclass="headerlink"href="#nilm-copy-wildcard"title="Permalink to this headline">¶</a></h2>
<p>Copy data and metadata, from multiple streams, between two servers. Similar to nilm-copy, except:</p>
<ulclass="simple">
<li>Wildcards and multiple paths are supported in the stream names.</li>
<li>Streams must always be copied between two servers.</li>
<li>Stream paths must match on the source and destination server.</li>
<li>If a stream does not exist on the destination server, it is created with the correct layout automatically.</li>
<h2><codeclass="docutils literal"><spanclass="pre">nilm-decimate</span></code><aclass="headerlink"href="#nilm-decimate"title="Permalink to this headline">¶</a></h2>
<p>Decimate the stream at SRCPATH and write the output to DESTPATH. The
decimation operation is described in Section 2.4.1; in short, every FACTOR rows
in the source are consolidated into one row in the destination, by calculating
the mean, minimum, and maximum values for each column. This program
detects if the stream at SRCPATH is already decimated, by the presence of a
decimate_source metadata key. If present, subsequent decimations take the
existing mean, minimum, and maximum values into account, and the output has the
same number of columns as the input. Otherwise, for the first level of
decimation, the output has three times as many columns as the input. See
also nilm-decimate-auto (Section 3.4.2.5) for a simpler method of decimating a
<dt>-u URL, --url URL</dt><dd>(default: http://localhost/nilmdb/) NilmDB server URL for the source stream.</dd>
<dt>-U DESTURL, --dest-url DESTURL</dt><dd>(default: same as URL) NilmDB server URL for the destination stream. If unspecified, the same URL is used for both source and destination.</dd>
<dt>-D, --dry-run</dt><dd>Just print intervals that would be processed, and exit.</dd>
<dt>-F, --force-metadata</dt><dd>Overwrite destination metadata even if it conflicts with the values in the “metadata” section below.</dd>
<dt>-s TIME, --start TIME</dt><dd>(default: min) Starting timestamp of data to decimate (free-form, inclusive).</dd>
<dt>-e TIME, --end TIME</dt><dd>(default: max) Ending timestamp of data to decimate (free-form, noninclusive).</dd>
<dt>-f FACTOR, --factor FACTOR</dt><dd>(default: 4) Set the decimation factor. For a source stream with n rows, the output stream will have n/FACTOR rows.</dd>
<dt>SRCPATH</dt><dd>Path of the source stream (on the source server).</dd>
<dt>DESTPATH</dt><dd>Path of the destination stream (on the destination server).</dd>
</dl>
</div><p>The destination stream has the following metadata keys added:</p>
<dlclass="docutils">
<dt>decimate_source</dt>
<dd>The source stream from which this data was decimated.</dd>
<dt>decimate_factor</dt>
<dd>The decimation factor used.</dd>
</dl>
</div>
<divclass="section"id="nilm-decimate-auto">
<h2><codeclass="docutils literal"><spanclass="pre">nilm-decimate-auto</span></code><aclass="headerlink"href="#nilm-decimate-auto"title="Permalink to this headline">¶</a></h2>
<p>Automatically create multiple decimation levels using from a single source
stream, continuing until the last decimated level contains fewer than 500 rows
total. Decimations are performed using nilm-decimate (Section 3.4.2.4).
Wildcards and multiple paths are accepted. Destination streams are automatically
named based on the source stream name and the total decimation factor; for
example, <codeclass="docutils literal"><spanclass="pre">/test/raw~decim-4</span></code>, <codeclass="docutils literal"><spanclass="pre">/test/raw~decim-16</span></code>, etc. Streams containing
the string “<codeclass="docutils literal"><spanclass="pre">~decim-</span></code>” are ignored when matching wildcards.</p>
<dt>-u URL, --url URL</dt><dd> (default: http://localhost/nilmdb/) NilmDB server URL for the source and destination streams.</dd>
<dt>-F, --force-metadata</dt><dd>Overwrite destination metadata even if it conflicts with the values in the “metadata” section above.</dd>
<dt>-f FACTOR, --factor FACTOR</dt><dd>(default: 4) Set the decimation factor. Each decimation level will have 1/FACTOR as many rows as the previous level.</dd>
<dt>PATH [...]</dt><dd>One or more paths to decimate. Wildcards are accepted.</dd>
</dl>
</div></div>
<divclass="section"id="nilm-insert">
<h2><codeclass="docutils literal"><spanclass="pre">nilm-insert</span></code><aclass="headerlink"href="#nilm-insert"title="Permalink to this headline">¶</a></h2>
<p>Insert a large amount of text-formatted data from an external source like
ethstream. This is a higher-level tool than nilmtool insert in that it attempts
to intelligently manage timestamps. The general concept is that it tracks two
timestamps:</p>
<olclass="arabic simple">
<li>The data timestamp is the precise timestamp corresponding to a particular row of data, and is the timestamp that gets inserted into the database. It increases by data_delta for every row of input. data_delta can come from one of two sources. If –delta is specified, it is pulled from the first column of data. If –rate is specified, data_delta is set to a fixed value of 1/RATE.</li>
<li>The clock timestamp is the less precise timestamp that gives the absolute time. It can come from two sources. If –live is specified, it is pulled directly from the system clock. If –file is specified, it is extracted from the input file every time a new file is opened for read, and from comments that appear in the files.</li>
</ol>
<p>Small discrepancies between data and clock are ignored. If the data timestamp ever differs from the clock timestamp by more than max_gap seconds:</p>
<ulclass="simple">
<li>If data is running behind, there is a gap in the data, so the timestamp is stepped forward to match clock.</li>
<li>If data is running ahead, there is overlap in the data, and an error is returned. If –skip is specified, then instead of returning an error, data is dropped and the remainder of the current file is skipped.</li>
<dt>-u URL, --url URL</dt><dd> (default: http://localhost/nilmdb/) NilmDB server URL.</dd>
<dt>-D, --dry-run</dt><dd>Parse files and print information, but don’t insert any data. Useful for verification before making changes to the database.</dd>
<dt>-s, --skip</dt><dd>Skip the remainder of input files if the data timestamp runs too far ahead of the clock timestamp. Useful when inserting a large directory of existing files with inaccurate timestamps.</dd>
<dt>-m SEC, --max-gap SEC</dt><dd>(default: 10.0) Maximum discrepancy between the clock and data timestamps.</dd>
</dl>
</div>
<i>Data timestamp</i>
<divclass="block-indent">
<dlclass="arglist">
<dt>-r RATE, --rate RATE</dt><dd>(default: 8000.0) data_delta is constant 1/RATE (in Hz).</dd>
<dt>-d, --delta</dt><dd>data_delta is provided as the first number on each input line.</dd>
</dl>
</div>
<i>Clock timestamp</i>
<divclass="block-indent">
<dlclass="arglist">
<dt>-l, --live</dt><dd>Use the live system time for the clock timestamp. This is most useful when piping in data live from a capture device.</dd>
<dt>-f, --file</dt><dd>Use filename and file comments for the clock timestamp. This is most useful when reading previously saved data.</dd>
<dt>-o SEC, --offset-filename SEC</dt><dd>(default: −3600.0) Offset to add to timestamps in filenames, when using --file. The default accounts for the existing practice of naming capture files based on the end of the hour in which they were recorded. The filename timestamp plus this offset should equal the time that the first row of data in the file was captured.</dd>
<dt>-O SEC, --offset-comment SEC</dt><dd>(default: 0.0) Offset to add to timestamps in comments, when using --file. The comment timestamp plus this offset should equal the time that the next row of data was captured.</dd>
</dl>
</div>
<i>Path and Input</i>
<divclass="block-indent">
<dlclass="arglist">
<dt>PATH</dt><dd>Path of the stream into which to insert data. The layout of the path must match the input data.</dd>
<dt>INFILE [...]</dt><dd>(default: standard input) Input data filename(s). Filenames ending with .gz are transparently decompressed as they are read. The default is to read the input from stdin.</dd>
</dl>
</div><divclass="admonition danger">
<pclass="first admonition-title">Danger</p>
<pclass="last">The following tools provide low level access to the NILM and are not
required for normal system use. Be careful running them as they may
corrupt the database or cause loss of data.</p>
</div>
</div>
<divclass="section"id="nilmdb-server">
<h2><codeclass="docutils literal"><spanclass="pre">nilmdb-server</span></code><aclass="headerlink"href="#nilmdb-server"title="Permalink to this headline">¶</a></h2>
<p>Run a standalone NilmDB server. Note that the NilmDB server is typically run
as a WSGI process managed by Apache. This program runs NilmDB
<dt>-v, --version</dt><dd> Print the installed NilmDB version.</dd>
<dt>-a ADDRESS, --address ADDRESS</dt><dd> (default: 0.0.0.0) Only listen on the given IP address. The default is to listen on all addresses.</dd>
<dt>-p PORT, --port PORT</dt><dd>(default: 12380) Listen on the given TCP port.</dd>
<dt>-d DATABASE, --database DATABASE</dt><dd>(default: ./db) Local filesystem directory of the NilmDB database.</dd>
<dt>-q, --quiet</dt><dd>Silence output.</dd>
</dl>
</div>
<i>Debug Options</i>
<divclass="block-indent">
<dlclass="arglist">
<dt>-t, --traceback</dt><dd>Provide tracebacks in the error response for client errors (HTTP status codes 400 - 499). Normally, tracebacks are only provided for server errors (HTTP status codes 500 - 599).</dd>
<dt>-y, --yappi</dt><dd>Run under the yappi profiler and invoke an interactive shell afterwards. Not intended for normal operation.</dd>
</dl>
</div></div>
<divclass="section"id="nilmdb-fsck">
<h2><codeclass="docutils literal"><spanclass="pre">nilmdb-fsck</span></code><aclass="headerlink"href="#nilmdb-fsck"title="Permalink to this headline">¶</a></h2>
<p>Check database consistency, and optionally repair errors automatically, when
possible. Running this may be necessary after an improper shutdown or other
corruption has occurred. This program will refuse to run if the database is
currently locked by any other process, like the Apache webserver; such programs
should be stopped first. This is run automatically on system boot for the Joule
<dt>DATABASE</dt><dd>Local filesystem directory of the NilmDB database to check.</dd>
<dt>-f, --fix</dt><dd>Attempt to fix errors when possible. Note that this may involve removing intervals or data.</dd>
<dt>-n, --no-data</dt><dd>Skip the slow full-data check. The earlier, faster checks are likely to find most database corruption, so the data checks may be unnecessary.</dd>
<dt>-h, --help</dt><dd>Print a help message with Usage information and details.</dd>
<dt>-v, --version</dt><dd>Print the installed NilmDB version. Generally, you should ensure that the version of nilmdb-fsck is newer than the NilmDB version that created, or last used, the given database.</dd>
</dl>
</div></div>
<divclass="section"id="nilm-cleanup">
<h2><codeclass="docutils literal"><spanclass="pre">nilm-cleanup</span></code><aclass="headerlink"href="#nilm-cleanup"title="Permalink to this headline">¶</a></h2>
<p>Clean up old data from streams, using a configuration file to specify which data
to remove. The configuration file is a text file in the following format:</p>
<spanclass="n">keep</span><spanclass="o">=</span><spanclass="mf">3.5</span><spanclass="n">m</span><spanclass="c1"># or 2520h or 105d or 15w or 0.29y</span>
</pre></div>
</div>
<p>Stream paths are specified inside square brackets (<codeclass="docutils literal"><spanclass="pre">[]</span></code>) and are followed by configuration
keywords for the matching streams. Paths can contain wildcards. Supported keywords are:
<dd>(default: automatic) Expected data rate. Only used by the <codeclass="docutils literal"><spanclass="pre">--estimate</span><spanclass="pre">option</span></code>. If not specified, the rate is guessed based on the existing data in the stream.</dd>
<dd>(default: true) If true, delete decimated data too. For stream path /A/B, this includes any stream matching the wildcard /A/B~decim*. If specified as false, no special treatment is applied to such streams.</dd>
</dl>
<p>The value keep is a maximum amount of data, not a cutoff time. When cleaning
data, the oldest data in the stream will be removed, until the total remaining
amount of data is less than or equal to keep. This means that data older than
keep will remain if insufficient newer data is present; for example, if new data
ceases to be inserted, old data will cease to be deleted.</p>
