Open Side Menu Go to the Top
Register
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** ** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD **

02-27-2019 , 09:18 PM
You could put whatever variables/parameters you want to pass along into a file and then have the child script source it. And there’s an easier way to do that code snippet you just posted, if I’m understanding what correctly what youre trying to do.

Do you have a parent script that calls another script and you want to share some parameters that parent script took as arguments?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
02-27-2019 , 09:19 PM
And then spend an hour trying to figure out how to get the value of the variables passed instead of the names of the variables...
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
02-27-2019 , 09:22 PM
If you tell me exactly what you’re trying to do I’m sure I could whip up a better solution

Shell isnt really intended to be used in that kind of parent/child class kind of paradigm imo
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
02-27-2019 , 09:32 PM
Every lamdba has its own script to invoke a cloudformation template which creates a CodePipeline job to build that lambda. An individual script looks like this:

Code:
#!/bin/bash

# creates or udpates the code pipeline and webhook that connects to this lambda
# build job runs buildspec.yml 

bash `dirname "$0"`/../../lambda-pipeline.sh \
  --action create \
  --reponame api-bl-donor_portal_user_lambda \
  --branch stage \
  --envname sandbox \
 "$@"
Oh yeah - relative file paths - also surprisingly hard.


lambda-pipeline.sh:

Code:
#!/bin/bash

# This script will create the base CodePipeline for node lambdas

# turns ** --name value ** paramaters into ** name=value **
while [ $# -gt 0 ]; do
  if [[ $1 == *"--"* ]]; then
    v="${1/--/}"
    declare $v="$2"
  fi
  shift
done

if [[ "$stackname" == "" ]]; then
  stackname=${reponame//*-/}
  stackname=${stackname//_lambda/}
  stackname=${stackname//_/-}
fi

if [[ "$envname" == "sandbox" ]]; then
  envsuffix="-sandbox"
fi

aws cloudformation $action-stack \
    --stack-name $stackname-$branch \
    --template-url https://s3-us-west-2.amazonaws.com/xxx-cfmplates$envsuffix/inf-cpl-resource_templates/lambda/github-webhook-lambda-pipeline.yaml \
    --capabilities CAPABILITY_IAM \
    --parameters \
      ParameterKey=RepoName,ParameterValue=$reponame \
      ParameterKey=Branch,ParameterValue=$branch \
      ParameterKey=BuildRuntime,ParameterValue=node
This is a common problem that has come up a bunch - this is one of the simpler examples. Basically I want a big generic script that does a lot of stuff and makes the actual aws-cli call, and little specific scripts that call the generic script - which may or may not include all of the variables (so some can be default). Maybe I need to graduate to something beyond bash.

Last edited by suzzer99; 02-27-2019 at 09:40 PM.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
02-27-2019 , 10:27 PM
Ah ok. I see what you are doing now.

In that case - I'd just have the children scripts write to a file somewhere and then source that file in the parent.


Code:
#!/bin/bash

# parent script

./child.sh -p1 "hello world"
source childparams.cfg
[[ $p1 == "" ]] && do_something
[[ $p2 == "" ]] && do_something_else
rm -f childparams.cfg
Code:
#/bin/bash
# child script

touch childparams.cfg

# parse args
while [[ $# -ge 1 ]]; do
    argv="$1"
    case $argv in
    -h|--help)
        # print_usage
        exit 0
        ;;  
    -p1|--param1)
        param1=$2
        echo "param1=$param1" >> childparams.cfg
        shift
        ;;  
    -p2|--param2)
        param2=$2
        echo "param2=$param2" >> childparams.cfg
        shift
        ;;  
    *)  
        echo "invalid parameter given to script"
        # print_usage
        exit 1
        ;;  
    esac
    shift
done

Last edited by jmakin; 02-27-2019 at 10:34 PM.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
02-27-2019 , 11:52 PM
"ehanced getopt" supports named parameters
https://medium.com/@codebyamir/parse...t-24a90b5b317d
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
02-28-2019 , 10:59 AM
open reg for .dev domains starts at 11 am
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-01-2019 , 09:36 PM
Man having a real, experienced product manager finally has made such a huge diff to my work/stress level. I was trying to fill in a lot of that role myself and it was super difficult. This guy actually speaks the same language i do and knows software engineering and what I need to deliver quality - it’s really such a relief. Like the upper management doesnt really understand my job role, it’s like something they knew they needed so they hired me but never thought past that. This guy’s such a lifesaver in that regard, by filling out my role a little more.

I know you guys are all mostly devs but i cant even describe to you the constant stress ive been under trying to do 3 different jobs, poorly, all at the same time. Please have pity on your managers.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-03-2019 , 04:12 PM
<waves>
I came back to 2+2 since I've started playing (live) poker regularly again, but this appears to be a much more relevant forum to my new career.

I became a data scientist in 2014-2015, but at my company (that I have now been at for 4 years, yikes) I spent more time in a data engineering capacity out of necessity, and for the last few years I have been an engineering manager for a team (recently: two teams) of both data engineers and data scientists. By their powers combined...

Good to see some OGs here still kicking around!
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-03-2019 , 09:44 PM
Welcome back!
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-05-2019 , 10:30 PM
I really feel like there should come a point when by far the most time consuming part of my job isn't figuring out how to sed/grep or otherwise bash some **** that is like basic programming 101.

I'm working through all this advanced AWS DevOps stuff. It's tricky but I'm getting through it. Oh great now I get back a JSON string as output from one aws-cli call that I need to pluck one value from as input to another aws-cli call. But it's fairly easy because I can just search for something like /"Address": "([^"])"/ - right?

That'll be an hour of endless stack overflow and confusion and 80 billion possible ways to do it for the simplest **** ever. I've found at least 20 different ways to do sort of what I want, but not really. So far.

https://stackoverflow.com/questions/...-regex-or-perl

And of course there's always a chance the stuff I'm doing on a mac doesn't work on ubuntu. I blame all of you *nix nerds for not revolting and demanding better.

This is exactly why I avoided becoming a linux guy for so long - just endless slog of command line arcanity. I think maybe running all my build jobs in a node environment and just doing this stuff with node scripts is a much better idea.

Last edited by suzzer99; 03-05-2019 at 10:43 PM.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-05-2019 , 10:33 PM
Quote:
Originally Posted by suzzer99
Oh great now I get back a JSON string as output from one aws-cli call that I need to pluck one value from as input to another aws-cli call. That'll be an hour of endless stack overflow and confusion and 80 billion possible ways to do it for the simplest **** ever.
I don't do much DevOps but I've seen some of our scripts using this, and it seems perfect for this

https://stedolan.github.io/jq/
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-05-2019 , 10:33 PM
I went from never having even touched command line anything in my life to relatively proficient, at least enough to pass by, in a month - just keep at it
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-05-2019 , 11:04 PM
I wish I had a command line, I’ve got 6 years of tech debt rolled into a CRM app whose GUI actively tries to make you screw up. After a year of working with this I get to graduate to copying and pasting Powershell commands in addition to the GUI... maybe they’ll give me access to a view into our SQL server tables where I can filter someone else’s stored procedures to research **** instead of a writing my own.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-05-2019 , 11:05 PM
I ended up with this:

Code:
aws rds describe-db-instances \
  --db-instance-identifier givedonor-wordpress-purple > temp.txt

newhost=$(cat temp.txt | grep -o '"Address": "[^"]*"' | sed 's/"Address": "//' | sed 's/"//')

rm temp.txt
Luckily this stuff is just to create a staging EB environment for a dev to make breaking wordpress DB changes in. So if it doesn't work he'll just come to me and say his environment failed to create. Otherwise I'd be a lot more nervous about that code.

For some reason I can't pipe the aws cli output straight into the variable and grep input or I get a BrokenPipe error - which you're not gonna believe this - but googling reveals an infinite variety of people pontificating on the possible causes and offering no real solutions.

I just found out another fun thing. If I edit my script while it's waiting for a command to finish - the edited script tries to run. It's apparently not loaded into memory at read time. At least I think that's happening.

Last edited by suzzer99; 03-05-2019 at 11:10 PM.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 12:26 AM
Definitely use "jq" for getting data out of json with bash.

ETA: or use a real language? You know AWS has APIs for everything right? java, python, probably a ton of others including javascript.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 12:27 AM
Do I have to install it on CodeBuild every time I run a job?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 01:16 PM


I hope for his sake that truck is unlocked.

A buddy of mine has a worthless neighbor who has tons of vehicles and recreational stuff like quad-runners that he's always working into the wee hours of the night making tons of noise. Also endless fence disputes and just terribleness all around.

The other night he asked me if I knew anyone who could hack him and ruin his life. I'm like I think you watch too many movies.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 02:59 PM
Quote:
Originally Posted by suzzer99
Do I have to install it on CodeBuild every time I run a job?
I don't know what a CodeBuild is, so I don't know. But the AWS command line tool you're using is a python script that uses the python library boto, so probably not?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 03:08 PM
Looking at that Bash stuff I would say, if you can do it in Python, do it in Python.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 03:12 PM
Quote:
Originally Posted by suzzer99
Do I have to install it on CodeBuild every time I run a job?
Yes but it should be pretty straightforward, in a pre_build step in your buildspec.yml:

Code:
phases:
  pre_build:
    commands:
      - echo "Installing jq..."
      - curl -qL -o jq https://stedolan.github.io/jq/download/linux64/jq && chmod +x ./jq
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 03:24 PM
why is a recruiter asking me to fill out a w4 before i am even offered a job? why would the recruiter need that anyways?

also, are there any situations where a recruiter would actually reduce your salary or be paid on-going beyond the initial hiring ?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 03:57 PM
Quote:
Originally Posted by OmgGlutten!
why is a recruiter asking me to fill out a w4 before i am even offered a job? why would the recruiter need that anyways?

also, are there any situations where a recruiter would actually reduce your salary or be paid on-going beyond the initial hiring ?
If a recruiter's pay is based on your salary you may think that they have a strong interest in you having a higher salary, but I think it would work like it does in Real Estate. You think the agents have an interest in a high price because that's how their pay is determined, but the difference in price is not nearly as important as whether or not deals actually happen. The agents represent the deal. So, if your recruiter thinks you are asking for too much and are less likely to get the job because of it, they would want you to ask for less. They just want to get paid.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 04:48 PM
ok, i asked them and i will be paid by the recruiting company itself, so i guess it's like a staffing company. this is for a few months only. normal i guess?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
03-06-2019 , 05:19 PM
Quote:
Originally Posted by well named
Yes but it should be pretty straightforward, in a pre_build step in your buildspec.yml:

Code:
phases:
  pre_build:
    commands:
      - echo "Installing jq..."
      - curl -qL -o jq https://stedolan.github.io/jq/download/linux64/jq && chmod +x ./jq
Yeah this might be worth it. But in general I'm trying to keep these down to a minimum - as that's one factor in how our builds eventually got to take 30 minute at my old job.

At some point I think I can use custom docker images for build jobs. But I haven't started looking into that yet.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote

      
m