The sandbox…
Atom links: https://atom.io/themes/atom-visual-studio-code-light-ui, https://atom.io/themes/native-ui
IoT inspector from Princeton: https://iot-inspector.princeton.edu/
Look into this backup software: https://www.cobiansoft.com/
That grep sed awk I’m always forgetting:
grep "Suspect" test_engine.log | grep -v "HTTP" | awk '{print $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18}' | sort > errors.txt
Rank ordering
On a handy ubuntu server, upload the text file and run this:
​tr -c ‘[:alnum:]’ ‘[\n*]’ < test.txt | sort | uniq -c | sort -nr | head -10
Where text.txt is the file name, and the -10 is the number of words to return.
See https://unix.stackexchange.com/questions/41479/find-n-most-frequent-words-in-a-file​.
There are ways to filter out extraneous words or elements: spaces, a, the, of, etc. but you need to pass in more arguments. Additional considerations are to ignore case.
Python regex search
# fileanalyser, analyses specified text file for patterns and prints out the line they occur in. # usage: cmd -> go to analyser file location # 'fileanalyser.py [path/to/specified/textfile.txt]' # the file is being analysed and a textfile with the logs is created after finishing (same directory as fileanalyser) import sys, os, re try: path = sys.argv[1] except IndexError: path = "-h" if path == "-h": print("\n\tusage: fileanalyser.py [path/to/textfile.txt]") exit() if os.path.exists(path): with open(path) as f: ctr = 0 previousLine = "" # regex' to check #find lines... #separated = re.compile("^.*-\n$") #... that end with a '-' plus = re.compile("^\+.*$") #... that start with a '+' whitespace = re.compile("^\s+([\S]+[\s]+)+$") #... that start with whitespaces (and are not empty) valueNoUnit = re.compile("^.*\d+$") #... that end with a number emptyLine = re.compile("^\s+$") #... that are empty brackets = re.compile("^[^\(]*\([^\)]*$") #... that contain an opening bracket but no closing one tempStr = "" #check every line for the specified regex strings above for line in f: ctr = ctr + 1 #if (separated.match(line)): # tempStr += "line " + str(ctr) + " separated\n" if (plus.match(line)): tempStr += "line " + str(ctr) + " starting with '+'\n" if (whitespace.match(line)): tempStr += "line " + str(ctr) + " starting with whitespaces\n" if (valueNoUnit.match(previousLine)): if (not(emptyLine.match(line))): tempStr += "line " + str(ctr-1) + "/" + str(ctr) + " value / unit separated\n" if (brackets.match(previousLine)): tempStr += "line " + str(ctr-1) + "/" + str(ctr) + " brackets separated\n" previousLine = line text_file = open("output.txt", "w+") #create output file if non-existent text_file.write("%s" % tempStr) text_file.close() else: print("File not found")
More coming soon….