The sandbox…

Atom links: https://atom.io/themes/atom-visual-studio-code-light-ui, https://atom.io/themes/native-ui

IoT inspector from Princeton: https://iot-inspector.princeton.edu/

Look into this backup software: https://www.cobiansoft.com/

That grep sed awk I’m always forgetting:

grep "Suspect" test_engine.log | grep -v "HTTP" | awk '{print $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18}' | sort > errors.txt

Rank ordering

On a handy ubuntu server, upload the text file and run this:

tr -c ‘[:alnum:]’ ‘[\n*]’ < test.txt | sort | uniq -c | sort -nr | head -10

Where text.txt is the file name, and the -10 is the number of words to return.

See https://unix.stackexchange.com/questions/41479/find-n-most-frequent-words-in-a-file.

There are ways to filter out extraneous words or elements: spaces, a, the, of, etc. but you need to pass in more arguments. Additional considerations are to ignore case.

Python regex search


# fileanalyser, analyses specified text file for patterns and prints out the line they occur in.
# usage: cmd -&gt; go to analyser file location
# 'fileanalyser.py [path/to/specified/textfile.txt]'
# the file is being analysed and a textfile with the logs is created after finishing (same directory as fileanalyser)

import sys, os, re

try:
path = sys.argv[1]
except IndexError:
path = "-h"

if path == "-h":
print("\n\tusage: fileanalyser.py [path/to/textfile.txt]")
exit()

if os.path.exists(path):
with open(path) as f:
ctr = 0
previousLine = ""

# regex' to check #find lines...
#separated = re.compile("^.*-\n$") #... that end with a '-'
plus = re.compile("^\+.*$") #... that start with a '+'
whitespace = re.compile("^\s+([\S]+[\s]+)+$") #... that start with whitespaces (and are not empty)
valueNoUnit = re.compile("^.*\d+$") #... that end with a number
emptyLine = re.compile("^\s+$") #... that are empty
brackets = re.compile("^[^\(]*\([^\)]*$") #... that contain an opening bracket but no closing one

tempStr = ""

#check every line for the specified regex strings above
for line in f:
ctr = ctr + 1

#if (separated.match(line)):
# tempStr += "line " + str(ctr) + " separated\n"

if (plus.match(line)):
tempStr += "line " + str(ctr) + " starting with '+'\n"

if (whitespace.match(line)):
tempStr += "line " + str(ctr) + " starting with whitespaces\n"

if (valueNoUnit.match(previousLine)):
if (not(emptyLine.match(line))):
tempStr += "line " + str(ctr-1) + "/" + str(ctr) + " value / unit separated\n"

if (brackets.match(previousLine)):
tempStr += "line " + str(ctr-1) + "/" + str(ctr) + " brackets separated\n"

previousLine = line

text_file = open("output.txt", "w+") #create output file if non-existent
text_file.write("%s" % tempStr)
text_file.close()
else: print("File not found")

More coming soon….