barrelsofdata/spark-boilerplate
Template
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
A boilerplate template for apache spark projects
14 Commits 1 Branch 0 Tags 229 KiB
Scala 100%
9d4d60aaa6
Go to file Use this template
karthik 9d4d60aaa6
Some checks failed
Build / execute-tests (push) Failing after 3m15s
Details
Tests / execute-tests (push) Successful in 4m38s
Details
Use gradle kotlin dsl, catalogs and update versions
2023-10-28 11:10:56 +02:00
.gitea/workflows Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
gradle Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
src Fix test run 2023-04-09 18:50:20 +02:00
.gitignore Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
build.gradle.kts Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
gradle.properties Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
gradlew Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
gradlew.bat Update spark and gradle versions, fix deprecated features 2023-03-16 12:32:45 +01:00
README.md Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
settings.gradle.kts Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00

README.md

Tests Build

Spark Boilerplate

This is a boilerplate project for Apache Spark. The related blog post can be found at https://www.barrelsofdata.com/spark-boilerplate-using-scala

Build instructions

From the root of the project execute the below commands

  • To clear all compiled classes, build and log directories
./gradlew clean
  • To run tests
./gradlew test
  • To build jar
./gradlew build

Run

spark-submit --master yarn --deploy-mode cluster build/libs/spark-boilerplate-1.0.0.jar