Skip to content

bervProject/crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Crawler Service

Current Implementation

  1. Read Badges Data from Credly to DynamoDB

Sample Data

Sample

Columns

Badge

{
  "id": "<badge id>",
  "name": "<badge title>",
  "src": "<image link>",
  "provider": "<provider name>",
  "expires": "<expires date time>"
}

Required Environment Variables

export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export AWS_REGION=
export DYNAMODB_TABLE_NAME=
export CREDLY_USERNAME=

TODO

  • Multiple Data Store Options (Extendable)
  • Custom Mapper Columns
  • Multiple Providers

License

MIT

About

Crawler Service

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •