4) sourceplot.py
sourceplot.py is a program to observe the datapoints created by sourcesub and stored in magcomparison. It can also update the outliers in the db, which can then be resubtracted using the -redo option in
Data Choosing Arguments:
-e, --epoch YYYYMMDD-YYYYMMDD
Enter the range of dates, or date to get the data from them. Not using this argument will choose to do it on all dates.
-n, --name OBJNAME
This option chooses the object name of the data you wish to view.
-m, --magnitude MAGLIST
Enter a list of magnitude to use for the graphs. If argument not used, the default is np.arange(14, 20, 0.5)
--filestr STRLIST
Enter a list of strings that are contained within the filename to improve searching. Can use for "e93", "fl.diff", or "optimal" or strings similar to that as well as a full filename.
-k, --keepout
Choosing this argument will keep the outliers in the plots you create.
-b, --bad
Choosing this argument will keep the data that has 9999 as a diffmag in your plots.
-u, --updateout
If you enter this argument, the
outliers will be updated for the magnitudes that you've chosen. I
suggest not doing this with the -b argument.
Graphing Tools:
-g, --graph GRAPHSTR
Where GRAPHSTR is a string with the type of graph you want to produce. There are 4 options: scatter, boxplot, histogram, and varstats. Scatter is the default graph if none are chosen.
scatter - produces a scatter plot
boxplot - produces a box plot
histogram - makes a histogram based on the counts of data within each bin. Idk how, but normalizing this data would be nice.
varstats - Tells you the mean and standard deviation of each magnitude chosen
--bestfit
This creates a best fit line and displays the slope and y intercept for the scatterplots. If it was possible to normalize the data in a histogram, it would be nice to have a fake normal curve placed over it, using this function. The best data will have a bestfit line similar to an y = x line.
--yx
This creates a y = x line on a scatterplot. Slope = 1, y int = 0
5) Other changes
I added the argument --filestr to around 7 files. For these files, I also needed to change Option Parser to Argument Parser in order to use the nargs option when parsing the arguments. This option allows you to enter a list of items, like "--filestr e91 e93" and will create a list like ['e91', 'e93']. This argument then passes these items down to query the filenames of data in photlco, photlcoraw, or magcomparison depending on where its used, and only chooses the files with any of the listed strings in them. This extremely useful for how much we have been manipulating filenames recently. Especially with David's optimal sub, and me changing the filename for the files that have a dropped source to have e93 instead of e91.
Along with this I also went and messed around with how some of the querying was being dealt with in mysqldef.py. I feel like a lot of it could even further be reduced and made better looking, but it is extremely long, and I feel like "If it ain't broke, why fix it?". The two functions I added to mysqldef were querytelescope, which adds onto the any query a way for searching for a telescope in a db, and querylike, which adds onto a query "datatable.column LIKE 'item'" where items are stored in a likelist, and datatable and column are based on a database.
Another thing I changed was one of Griffin's recent changes with suffix. I use the same suffix creation in sourcesub.py with filestr to only perform actions on certain images. Because we are using it twice, i made it into a function called difference_suffix, which is stored in myloopdef.py, for easy access.
Another thing I was thinking might be useful is to incorporate the print_and_run_command function that I have in sourcesub.py. It would save a lot of space, because printing then running commands are done multiple times throughout the pipeline.