In this YouTube series I am covering 1. Cloud 101 2. Getting started with AWS 3. Introduction to AWS IAM 4. Introduction to AWS Storage Services 5. Introduction to AWS Networking Services 6. Introduction to AWS Migration Services ( Data Migration Services) 7. Introduction to AWS Compute Services – EC2 and Lambda 8. Introduction to… Read More AWS Learning series for Beginner
In this blog we are going to learn how to you improve query performance of you Glue ETL Job by adding parallelism in your database JDBC connection. If you are running an AWS Glue ETL Job for a large database table and if you are running OOM (Out of Memory error message) because all the… Read More AWS Glue : How to perform JDBC read parallelism
Error Message :- When I was working with AWS Glue Interactive session, I got an error User arn:aws:iam::<$aws-account-id>:role/AWSGlueServiceRole-glueworkshop/GlueJobRunnerSession is not authorized to perform iam:PassRole on recsource arn:aws:iam::<$aws-account-id>:role/AWSGlueServiceRole-glueworkshop because no identify-based policy allows the iam:PassRole action. However, you may receive similar error message while working with other services too. Before we move with resolution, let’s understand… Read More How to Resolve iam:PassRole error message?
In this Video – We are going to learn What is AWS Account Alias ? Why someone need to Create AWS Account Alias ? How to create AWS Account Alias
AWS Glue – AWS Glue is a serverless ETL tool developed by AWS. It is built on top of Spark. As spark is distributed processing engine by default it creates multiple output files states with e.g. Generating a Single file You might have requirement to create single output file. In order for you to create… Read More Create single file in AWS Glue (pySpark) and store as custom file name S3
There is no other time better than this when Nasa and SpaceX successfully launched their first Astronauts to space station from U.S. since 2011. This launch will create lots of new opportunities in space science and will create a new race in space dominance. When all these developments are going on, we cannot find better… Read More Tech Talk with Saurabh Kaushal ( TEDx speaker)
Step 1 – Install MS-SQL on your Mac Microsoft has made SQL 2017 and later version available for MacOS and Linux system. This change has made possible to run SQL Server from a Docker container. There for no need to install a virtual machine with Windows. Follow instruction described in below article to install MS-SQL… Read More Installing MSSQL on MAC
Issue – while deploying Serverless Lambda (sls deploy command ) function I got below error message An error occurred: EsLambdaFunction – The provided execution role does not have permissions to call CreateNetworkInterface on EC2 (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: xxx). Resolution – In order to resolve the issue; I have… Read More Serverless – The provided execution role does not have permissions to call CreateNetworkInterface on EC2
Error Message – User: arn:aws:iam::xxxx:user/admin is not authorized to perform: lakeformation:PutDataLakeSettings with an explicit deny Issue – This issue will occur when you will try add new user as Lake formation admin with IAM user which already added as admin to Lake formation lakeformation:PutDataLakeSettings Denying this permission prevents a data lake administrator from designating other… Read More AWS Lake formation – Error Message
Feature Comparison – Aurora PostgreSQL vs MySQL Feature Comparison As of Sep 2019. Features Features details Aurora PostgreSQL Aurora MySQL RDS PostgreSQL High Availability Read-replica Yes ( Up to 15 ) with sub-10ms replica lag Yes ( Up to 15 ) with sub-10ms replica lag Yes ( Up to 5 ) cross-region read replicas No… Read More Aurora PostgreSQL vs MySQL Feature Comparison
Ensure you quickstart docker vm is running ( you can check by using below command) https://hub.docker.com/r/cloudera/quickstart/ Launch VM container ( Go to terminal and run ) docker run –hostname=quickstart.cloudera –privileged=true -t -i -v /Users/navnit/Documents/cloudera:/src –publish-all=true -p 8888 cloudera/quickstart /usr/bin/docker-quickstart 2. Run docker ps command to validate if cloudera/quickstart is running… Read More How to copy file from cloudera quickstart docker to local file system ( mac or windows )
Step 1 – Connect to your ec2 instance ( follow below article ) https://learnsqlteam.com/2018/11/30/how-to-connect-aws-ec2-instance-through-terminal/ step 2 – go to /etc – ( cd /etc ) step 3 – edit crontab ( nano crontab ) step 4 – define schedule and command e.g. in this example I am sysning file b/w ec2 instance and s3 bucket… Read More How to enable cron schedule in AWS EC2 instance
Step 1 – Ensure your EC2 instance is running Step 2 – Get EC2 instance name or public ip . Step 3 – Locate your Key Pair ( you must need keypair to connect to AWS services programmatically ) Step 4 – grant sufficient access chmod 400 myKeypair.pem Step 4 – ssh to your ec2… Read More How to connect AWS EC2 instance through terminal
input [a,b,c,d] [b,c] [f,g,h] output a b c d b c f g h with NS AS ( select 1 as n union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9… Read More Redshift – How to flatten JSON data in redshift
Learn Sql Team 