What do you automate in your life with your programming skills?
29 08 Dec 2015 05:49 by u/PlasmaDistortion
I have been inspired by the article about the programmer that automated many things in his life. Whether or not it is true is another story, but I think it is good motivation for us to see how far we can push things. I have personally started with some basic things like file migrations and log collections but I am looking to expand it so I can see how far I can push it.
What I would like to know is how you have personally automated things either with your job or daily life. What would you like to automate but have not yet take the time to?
47 comments
14 u/Silver_Tube 08 Dec 2015 14:46
I work IT for a bank. They don't treat their employees well. There is a lot of turnover. Management has required that everyone's job functions be listed and documented, everyone has been reduced to robots that follow checklists. This is to avoid what they call the "key person vulnerability". It has the side effect of sapping everyone of any problem solving skills. Any time something off script happens they call me.
I studied computers but find myself solving problems like ach transfers not posting, mysterious fees showing up on people's statements, credit reports containing inaccurate information, money going to middle eastern countries, etc. I get treated like a lowly computer monkey and it is frustrating, but that is life if you want to keep a job I suppose. It's better than living in the parents basement.
I use Python scripts to automate the jobs of people who piss me off at work. Since I already solve all their problems anyway, I can just open up their checklists and make scripts that perform their regular functions and put them on a schedule. Most of these people are just downloading files from vendors and putting them into different folders on the network, then firing off jobs in our financial core to parse the information. My scripts are better at it than they are. The sad thing is, people get really happy when I make one of these scripts for them, nobody has caught on to the fact that I'm automating them out of existence.
These programs aren't particularly advanced. Parsing text files ad copying things across samba shares, real beginner level stuff. You can do a lot of work with just that. I use python. If you want to get into it I suggest you star accumulating what I call a "code pallete" which is just a big file of commonly used code snippets for typical tasks that you can copy from to make your program.
Some suggestions for snippets you'll want to have:
Command line argument parsing and validation, using glob and path expansion (that is, * for capturing all files, ~ for referring to the home folder, etc)
Copying files between servers on a network(this is usually Windows samba shares, pretty much every language has a nice module for this available)
Sending emails (to report task completion, job failure, etc)
Connections to ftp, ssh, telnet servers
Ping (for monitoring connection status of remote systems)
Mind you, this is all beginner level stuff, we haven't even gotten into proper data manipulation, I'm still trying to learn that part, but you can do a lot with web page scrubbing and database connections and stuff. I'm looking into the selenium kit, it automates web page interaction, I could replace a lot of my annoying users if I was good at that.
2 u/Omnipresent 09 Dec 2015 02:30
Is there any reason you use python in particular?
2 u/Silver_Tube 09 Dec 2015 02:52
There are lots of modules for it
I find the language easier than others to look at code and figure out what it's doing at a glance. That's probably subjective.
There's blogs and tutorials and videos aplenty that explain anything I'm likely to be curious about.
The local user group is very smart and helpful.
I like a bunch of different languages but for 'ease of development' Python is my favorite of those that I've tried.
2 u/GumbyTM 09 Dec 2015 05:13
It's creepy how similar my job is to your description. With a couple of minor exceptions, most notably, I like where I work, am treated quite well, and use different tools our jobs are virtually identical. I work for a bank and like to call myself the digital janitor. I get everything from data transfer projects to missing trade data and even "I forgot the password to my excel file.".
Anyway, a couple things really struck home and I thought I might offer some advice if you wish to take it...
Your job is a lateral, and as such you will most likely have persistent access to key personnel. Use excuses like data verification, signoffs and whatever else to get as much time with these people. Build relationships and work on your people skills. You skill set will make it incredibly easy to pivot into other areas and even industries.
Like the lateral personnel issues, use the same advantage to learn as much as possible about the industry. A digital skill set coupled with things like a deep knowledge of the new (TRID|AML|UMDAA|etc..) laws make you a very desired person.
Don't just replace, but make the argument for the ROI of your work. Suggest that if some of the freed resources were allocated to your department it could be leverage to even greater gains. This will place you in a likely position to acquire subordinates and grow your team.
And finally....
A: Users in Servicing entered bad information.
A: Users in Credit entered bad information
A: Users in the branch entered bad information.
Best of luck,
Fellow evil banker
1 u/Silver_Tube 09 Dec 2015 16:56
I'm curious about what sort of different tools you are using. Nobody is really teaching me any of this stuff and I'm guessing my way through it.
I try to avoid the folks upstairs as best I can, but I hear you about relationships. These people don't really understand the amount of weight I'm carrying for them and the only way they ever could is if I spend time with them.
Sometimes it feels like they go out of their way to keep us in the dark about the proprietary stuff, I've heard a couple of those acronyms before.
0 u/GumbyTM 11 Dec 2015 01:27
Most of our quick deployment stuff is done with MS Office suite tools, Crystal, and some proprietary software. I would punch a puppy for my own server with PHP and MySQL but our IT is run by a company which is a subsidiary of ours and the rules governing banks and subsidiaries mean we can't tell them to do squat so changes and access requests take weeks. As a result we do as much in house as possible which means heave reliance on the MS Suite. The MS tools are nice in the sense that everyone has access to them and most people have at least a rudimentary skill set with these products. They also allow us to produce proof of concept items extremely quickly which can serve as a stop gap while our IT departments hobbles along.
Decisions on tools are based on need and development time and can be as simple as a macro enabled spreadsheet or changes published to our intranet. Updates to our proprietary systems are done with the built in tools or are kicked back to the vendor if needed.
Unfortunately you'll probably find this is the case for the rest of your life. No one knows what they should and everyone else is faking it to some degree as well. The trick is to look like you know what you are doing until you actually do.
Once competent, people will drop more crap on you to stress about. The only way to succeed is to view these challenges as opportunities and to leverage them to advance your career.
One day, long from now and long overdue someone will acknowledge your work and effort. Those days are few and far between but that's all you get.
All jobs suck to a degree, that's why they pay us to go there.
Stay positive, fight for knowledge and empathize with others. Likeability will advance your career as much as skills. Hope this helped.
7 u/Ohooh 08 Dec 2015 08:47
I'm not even a programmer, just saw this on the front page. But a few years back I worked a job where I did quality control for a transcription company. My job was to keep things uniform (formatting, slang words) and correct punctuation, grammar, and spelling mistakes.
I was able to make a fairly basic Microsoft Word macro that would do all of the formatting as well as fix about 30 of the most common mistakes. I worked in a small, fairly amateur office and my boss had no idea how much that streamlined the work, so I never let him know and just used it to give myself as much free time throughout the day as I wanted.
6 u/anoneko 08 Dec 2015 07:03
Collecting pics from websites en masse and automatic renaming/relocating them when saving based on picture tags.
1 u/Morbo 08 Dec 2015 08:40
Found the lazy but efficient porn collector.
11.jpg -->
string newFileName = Guid.NewGuid().toString() + '.jpg';4 u/Scine 08 Dec 2015 21:03
Built a VMWare host with 2 Linux VM's to pull TV shows down from various points, move them to their respective folders, rename them, and do a refresh on my Plex server which I view with my Roku 4k box.
Software needed:
NZBGet Sonarr Plex Couch Potato Deluge
Some scripting on my part to get it all to work perfectly.
Now, instead of a $140 DirecTV bill, I pay $7 a month for Usenet account, $8 a month for Netflix, and my company pays for my Amazon Prime account.
I did ALL this because my wife was in the hospital waiting for our baby to arrive (he's fine and so is she...precautionary), and she wanted to watch Big Bang Theory. CBS wanted me to be a member of their site (fine), but they wanted $12 a month to watch the last episode aired.
Fuck that.
So, I took the better part of 3 days getting it dialed in JUST right, and now I've turned that hobby project into "This is going to replace DirecTV as soon as my contract is up".
I don't watch sports, I don't want to pay for sports channels, and I sure as shit don't want to pay $140 a month for all of 10 channels.
3 u/kltpzyxm 08 Dec 2015 06:14
Working on a wifi coffee pot.
1 u/PlasmaDistortion [OP] 08 Dec 2015 07:55
I will be watching for it on Kickstarter.
3 u/kltpzyxm 08 Dec 2015 23:33
No kickstarter, but it will be open source.
3 u/Heru 08 Dec 2015 06:54
Lights, coffee, brightness of screens, dev environment vs play environment (opening desktop apps based on location), temperature of home, locking / unlocking of computer based on proximity, clearing of browser history based on proximity to beacons, builds of software, and much more.
2 u/PlasmaDistortion [OP] 08 Dec 2015 07:56
What do you use to unlock your machine by proximity? This would be an awesome one to implement.
2 u/Heru 08 Dec 2015 14:20
Bluetooth LE beacon in my pocket detected by my computer
3 u/crankypants15 08 Dec 2015 15:56
I wanted a program I had more control of so I could make EPUB books from Markdown. So I did it.
I started out at the www.pgdp.net project, for proofreading public domain books which become EPUB books and other formats. But the books were so damn BORING and pointless beyond any people with a super-specialized hobby or interest. Thousands of hours go into books that almost no one will ever read. So I wanted to convert my own books into EPUB. And here I am.
In my work life I have about 15 reports for execs that just run automatically. Some run daily, some weekly, some monthly. Once the program works, my work stops there. It's literally getting tons of things done without lifting a finger.
0 u/GoodGodKirk 09 Dec 2015 04:55
sensusaccess.com, zamzar.com, or try calibre.
3 u/DinoRider 08 Dec 2015 19:59
I automate a lot of Linux system installation / setup tasks. My main tools are Puppet to automate configuration on mulitple systems and backup / restore commands for data and most of my own user config.
Every time I setup a new system I marvel at how much time I used to waste typing in package install commands and editing config files and copying things manually from backups. Once I have the basic OS and networking setup I only need to check on it a few times while the machines do hours of work by themselves.
On a less technical note, don't underestimate the value of a good task / reminder system to get things out of your head so you can focus on the present without worrying about forgetting things. I personally use Toodledo and Google Calendar, but there are a lot of ways to manage your life that are far better than the ad hoc disaster some of us run by.
2 u/dragon_-_son 08 Dec 2015 07:12
Actually had start to make basic scripts for what I have to do at my new job. They have windows 7 pro running on 1.27Ghz dual core with 1Gb of RAM, with some random antivirus I've never heard of, and screen monitoring and call recording software. At any given time it's using 70-97% of the system resources with Firefox with between 2 and 8 tabs open, outlook, and notepad. I'm literally waiting on my computer to do more than 1 thing at a time. So I'm making scripts to start most of what I need to do. Then I just wait until the computer catches up. I asked the IT guy if I could just upgrade the damn thing myself since I know how, and he said no just put in an IT ticket for it. Been a month and nothing. Thinking I'm going to be upgrading it myself and not telling anyone.
0 u/PlasmaDistortion [OP] 08 Dec 2015 07:58
That sounds freaking miserable! Take it a step further and just kill all that crap that is running. It will probably piss off the IT guy but screw him. He sucks at his job.
1 u/tame 08 Dec 2015 08:17
Doesn't really count as "programming skills" but reticulation for the garden runs on a timer, and we've got a few lights and things running on timers/sensors so they turn on and off as required.
On my desktop (Win 8.1) I've found a huge productivity boost from installing Launchy. It's great being able to load any program or file with a few keystrokes.
Currently working on learning Android development and the Pebble protocol so I can start doing cool things with my Pebble watch.
1 u/PlasmaDistortion [OP] 08 Dec 2015 08:23
Yep, Launchy is great. I have used it on and off for a couple of years.
1 u/Techttz 08 Dec 2015 14:36
Reminders man, reminders everywhere.
1 u/supernatendo 08 Dec 2015 19:08
I automate the shit out of things. When I'm on Linux, I automate in Bash, at work I've taught myself powershell.
Yesterday I wrote a power-shell script for manipulating an excel document some crazy accountant wrote to automate a proprietary piece of software we have to use. Working with that excel document is hellish. It is working working pretty nicely and it should help a lot at work, but my coworkers seem very unsupportive of my efforts.
I think a lot of my coworkers are worried about automation taking away the boring repetitive tasks we do every day. Some people think those boring repetitive tasks will keep them employed forever. They see automation as a threat. I see automation as a way for us to remove the shackles of drudgery and pursue more entertaining and fulfilling careers.
1 u/freddymercury 09 Dec 2015 04:17
I automate mapping and labels with gis and civil 3d
1 u/NiklausTheNaked 09 Dec 2015 05:55
I made a simple PHP script that operates an APC master switch to turn on/off various electrical gadgets. I really only use it for a single lamp, though.
1 u/Entomo 12 Dec 2015 03:27
I consolidate all my pictures (DLR and several family phones), rotate them according to the camera sensor, and store them in directories by date/time taken. Then I print out a list of what's left in the source directory (usually empty directories and a movie or two). I use jhead and bash.
I also scripted a git check-in of my mGSD tiddlywiki. which has all my work and home TODO stuff in it.
When developing, I usually just use the up-arrow in the CLI (Linux) to repeat similar commands unless it's something I don't do frequently. Things like "clean and compile code, create jar file, move jar file, restart webserver" can all be done on one line - storing that in bash history is a sort of automation, I guess.
1 u/redditor1255 16 Dec 2015 19:14
My server has an active hard drive and a backup. To protect the backup, it is kept unmounted. Once a day a cron job mounts the backup, rsyncs the files across, then unmounts the backup.
If my laptop is left on over night and is connected to the server, it bzips my home folder minus a few things over to the server. I need to update the script to delete backups over a month old.
At work I have scripts that pull diagnostic data and dump it to my screen. I maintain a very specialized system, and figuring out wtf went wrong can be time consuming. This saves me from having to dig around in endless folders.
So really just the basic stuff.
0 u/BASH_SCRIPTS_FOR_YOU 10 Dec 2015 07:35
I automatically download porn with BASH