Bash script, get temperature digits but also, keep a more formal copy?

June 24, 2022 3:10 PM

I'm working on a long bash one-liner that prints out the current temp in F's where I live. I like that it just prints out a number, which is helpful. But what's a good way to also keep a dated record of the number, and then just print out the number itself? Script & details inside.

curl -sS "https://api.openweathermap.org/data/2.5/weather?q=mycity&units=imperial&appid=apikeyhere" | jq '.main.temp' | awk '{print int($1+0.5)}'

Just to save anybody's time here, the awk part at the end is a way I just learned to round the number instead of getting 97.22 or whatever.

What I'd like to add is this kind of feature:

1. Append the ISO date like "YYYY-MM-DD," and then this temperature number, to a CSV file named YYYY-MM-DD.csv so we can have a casual record to play with.

2. Then just print out this number so I can put it in my fish prompt or whatever.

The API is fun to play with but this is mainly a local scripting question.

Alternative or just neat approaches to my JSON piping here are also welcome as I'm enjoying learning to be a kind of scripting/programming generalist.

posted by circular to Computers & Internet (12 answers total)

Best answer: Sure, first you want to store the output of your pipeline above into a variable. Eg:

temp="$(curl .... | .... )"

Now you can output that to stdout like it was doing:

echo "$temp"

I'll refer you to the date command to get the date formatted like you want (see "man date" for the gory details). You can just assign the output of that command to another variable, using the same method shown above.

Then to write/append to your csv files:

echo "$date, $temp" >> "$date.csv"

posted by joeyh at 3:36 PM on June 24, 2022 [5 favorites]

Best answer: This is a good place for the tee command. Something along the lines of:

#do stuff here to populate $date; echo -n "$date, " >> $date.csv; curl -sS "https://api.openweathermap.org/data/2.5/weather?q=mycity&units=imperial&appid=apikeyhere" | jq '.main.temp' | awk '{print int($1+0.5)} | tee -a $date.csv'
posted by hanov3r at 3:40 PM on June 24, 2022 [2 favorites]

Best answer: Actually, to clean that up:

DATE=`date "+%Y-%m-%d"`;echo -n "$DATE, " >> $DATE.csv; curl -sS "https://api.openweathermap.org/data/2.5/weather?q=mycity&units=imperial&appid=apikeyhere" | jq '.main.temp' | awk '{print int($1+0.5)}' | tee -a $DATE.csv

If you're running this multiple times per day, you could always add TIME=`date "+%H:%M:%S" at the start and then write $TIME to your file instead of $DATE.
posted by hanov3r at 4:17 PM on June 24, 2022 [2 favorites]

I'd suggest using a script with multiple lines, for readability and easier future modifications and comments and the like.

Then, you could check if today's temperature was already recorded, or whatever interval you decide on.
posted by Pronoiac at 8:21 PM on June 24, 2022 [1 favorite]

Best answer: here are some assorted ideas for extension exercises, in arbitrary order.

a) figure out how to "harden" the error handling logic around the http request in your script to deal with possible real-world error scenarios like -- what happens if the weather data server is temporarily unavailable or broken and returns a http 500 or http 503 response without any temperature data in the response body? what if the server is dead and doesn't reply at all? what if there is a network problem that causes the server not to get your request -- how long should your script wait for a reply? what should it do if it fails to get a reply? curl options for --fail and --retry could be interesting. see also bash options for setting different error handling modes: set -e, set -u, set -o pipefail

b) configure your operating system to periodically run the script. if you run linux, maybe cron or perhaps something like systemd timers could work

c) rewrite the script to be a single python3 script, without delegating work to command line applications such as bash, curl, jq, awk. Here are some interesting standard library modules: csv, urllib.request, json

d) rewrite the script to be a single go (golang) application, again without delegating work to command line applications. Interesting standard library packages include net/http, encoding/json, encoding/csv

e) for a data storage file format, replace the use of CSV with a sqlite database. python offers a built-in sqlite module. go doesn't offer built-in sqlite library but there are open source go libraries that wrap a sqlite driver, here is an interesting tutorial that shows how to use it

f) get access to one or more computers that are different to the computer you used to develop the script, try to get different versions of the script working reliably on all these other computes. is it easier to install and get the script running on a fresh computer if the script is built out of bash, out of python, or out of go? why?

f2) learn how to use a configuration management tool like ansible to automate the installation of your script to multiple computers, so the script and additional runtime dependencies required (if any) can be installed across multiple machines without any manual steps.
posted by are-coral-made at 3:00 AM on June 25, 2022 [1 favorite]

oh! if you are writing shell scripts, it is _mandatory_ to install and learn to use the shellcheck tool to detect errors and bad style in your scripts. "ShellCheck finds bugs in your shell scripts."
posted by are-coral-made at 3:08 AM on June 25, 2022 [2 favorites]

Best answer:

echo 99.5 | awk '{print int($1+0.5)}' | while read t; do echo "$(date),$t" >> log; echo $t; done

# or

echo 99.5 | awk -v date="$(date)" '{temp=int($1+0.5); print date,",",temp >> "log"; print temp;}'

# or

echo 99.5 | awk 'BEGIN { d = "date"; d | getline date}{temp=int($1+0.5); print date,",",temp >> "log"; print temp;}'

You'll need to mess with the `date` command to get your preferred format. You might have to futz with the awk part a bit (it's been so long since I've used it that I don't remember the string concatenation specifics).

I'd use Perl :)
posted by zengargoyle at 4:23 AM on June 25, 2022 [1 favorite]

Response by poster: These are great answers. Thanks everybody for the helpful responses, I'm looking forward to learning more.
posted by circular at 12:01 PM on June 25, 2022

There is a zen sort of beauty in just typing out a one liner to do something. It's a worthy task. (and quite fun).

Tips if I have any....

Make sure your shell has long scroll back history. Many script start with `history >> HISTORY` and then you edit things down to the things that worked and then you generalize and turn it into a script or a function.

When working on something, be in a directory, and if nothing else, cut and paste that woenderful command into a file. I call mine '0liners'. Bonus, format it like a `fortune` file (single '%' on a line delimits records). Keep it consistent. Then for lol's and giggles you can find and smush all of your '0liner' files into a single file and plop it into your login fortune file and go "OMG WTH does that even do?".

It's only the second or third time that you look up through your '0liners' that you really load it up into an editor and fix it up so you can put it somewhere and tell people to do it themselves.

If you're using *NIX, the `fc` command will open your last command in `$EDITOR`. Save and Exit. (If things get long enough to need editing). After that.... turn into actual script.

There is no shame in busting out one-liners, just keep them. If it's a recurring thing.... script them up and document them and tell whomever to do it themselves and stop bothering you with trivial things.

I would second `shellcheck` for pure 'sh' scripts. The same goes for anything else. I mostly used Perl, but there's `perltidy` to format things and `perlcritic` that point out bad code, you can configure those tools.

I also keep a '~Template/foo' like directory of bits of code available to include during the one-liner to script process. Standard headers, option processing, documentation template, etc. It's a mix-and-match sorta thing.

Things go from one-liners to to a larger more standardized documented thing. But don't waste that time for the one-off simple thing.

Make a new directory for each more than minor thing, named with the date/project/case whatever. Work there and keep a 'History' and '0liners' in there just in case you need to do it again.

I also keep a '~/Notes' directory with sub-folders full of just interesting tidbits of code examples ripped from man pages or documentation or whatnot. `grep` is your friend. like the '0liners' and '~/Templates' and now '~/Notes' and your directory of past projects.... You can use `grep` or something similar to find everything you think you know because you've done it before.
posted by zengargoyle at 3:39 PM on June 25, 2022 [1 favorite]

Another good thing about multi-line scripts: posting them on MetaFilter won't lead to weirdness in threads or Recent Activity due to a too-wide column
posted by Pronoiac at 8:29 PM on June 26, 2022

Funny, I was noticing that long lines didn't mess up things. Mine showed up with a horizontal scrollbar beneath it. My biggest issue posting code is remembering to do the HTML Entity escaping. There's often a missing '<' somewhere or things are a bit messed up after because it ran afoul of HTML tag stripping. You don't have to just test your answer, you have to check that things got cut-n-pasted correctly. Bit of a PITA.

Long lines of code don't mess with my browsing experience. But I wouldn't put it past causing other users trouble in some way.
posted by zengargoyle at 11:24 AM on June 28, 2022

I thought to check: they're wrapped in the Modern themes, but result in wide, mostly blank pages in the other two.
posted by Pronoiac at 10:42 PM on June 28, 2022