barrelsofdata/spark-boilerplate
Template
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
A boilerplate template for apache spark projects
16 Commits 1 Branch 0 Tags 229 KiB
Scala 100%
main
Go to file Use this template
karthik 070a7c7c1b
All checks were successful
Tests / reset-status (push) Successful in 2s
Details
Tests / tests (push) Successful in 4m50s
Details
Tests / build (push) Successful in 3m42s
Details
Switch to single workflow file
2023-10-28 11:34:28 +02:00
.gitea/workflows Switch to single workflow file 2023-10-28 11:34:28 +02:00
gradle Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
src Fix test run 2023-04-09 18:50:20 +02:00
.gitignore Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
build.gradle.kts Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
gradle.properties Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
gradlew Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00
gradlew.bat Update spark and gradle versions, fix deprecated features 2023-03-16 12:32:45 +01:00
README.md Switch to single workflow file 2023-10-28 11:34:28 +02:00
settings.gradle.kts Use gradle kotlin dsl, catalogs and update versions 2023-10-28 11:10:56 +02:00

README.md

Tests Build

Spark Boilerplate

This is a boilerplate project for Apache Spark. The related blog post can be found at https://www.barrelsofdata.com/spark-boilerplate-using-scala

Build instructions

From the root of the project execute the below commands

  • To clear all compiled classes, build and log directories
./gradlew clean
  • To run tests
./gradlew test
  • To build jar
./gradlew build

Run

spark-submit --master yarn --deploy-mode cluster build/libs/spark-boilerplate-1.0.0.jar