Serverless – The provided execution role does not have permissions to call CreateNetworkInterface on EC2

Issue – while deploying Serverless Lambda (sls deploy command ) function I got below error message An error occurred: EsLambdaFunction – The provided execution role does not have permissions to call CreateNetworkInterface on EC2 (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: xxx). Resolution – In order to resolve the issue; I have… Read More Serverless – The provided execution role does not have permissions to call CreateNetworkInterface on EC2

Replication – Agents Job Location

Issue – Where Replication Agents job Locate Resolution – While troubleshooting replication issue; it’s always good to know which server have which job. In transnational replication scenario. Snapshot agent Log Reader agent Distribution agent If Publisher and Distribution Server is same and you have configured Push subscription; all the job reside Publisher\Distributor server. If Publisher and… Read More Replication – Agents Job Location

Replication Issue – Cannot execute as the database principal because the principal “dbo” does not exist, this type of principal cannot be impersonated, or you do not have permission.

Issue – Replication Log Reader agent was failing with below error messages. 2018-04-24 19:33:07.854 Status: 32768, code: 53044, text: ‘Validating publisher’. 2018-04-24 19:33:07.873 Status: 4096, code: 20024, text: ‘Initializing’. 2018-04-24 19:33:07.873 The agent is running. Use Replication Monitor to view the details of this agent session. 2018-04-24 19:33:07.879 Status: 0, code: 20011, text: ‘The process… Read More Replication Issue – Cannot execute as the database principal because the principal “dbo” does not exist, this type of principal cannot be impersonated, or you do not have permission.

Reading\Writing Different file format in HDFS by using pyspark

Issue – How to read\write different file format in HDFS by using pyspark File Format Action Procedure example without compression text File Read sc.textFile() orders = sc.textFile(“/user/BDD/navnit/data-master/retail_db/orders”) Write rdd.saveAsTextFile() orders.saveAsTextFile(“/user/BDD/navnit/saveTextFile/orders”) sequence File Read sc.sequenceFile(ordersSF = sc.sequenceFile(‘/user/BDD/navnit/saveSequenceFile/orders’) Write PipelinedRDD.saveAsSequenceFile() ordersKV.saveAsSequenceFile(‘/user/BDD/navnit/saveSequenceFile/orders’) Avro file Read sqlContext.read.format(“com.databricks.spark.avro”).load() orders = sqlContext.read.format(“com.databricks.spark.avro”).load(“/home/BDD/navnit/orders/”) Write dataFram.write.format(“com.databricks.spark.avro”).save() orders.write.format(“com.databricks.spark.avro”).save(“/user/BDD/navnit/saveAvroFile/orders”) Parquet File Read sqlContext.read.parquet() ordersParquet =… Read More Reading\Writing Different file format in HDFS by using pyspark