-
Notifications
You must be signed in to change notification settings - Fork 51
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Some time ago (not sure exactly when), Amazon updated their Kindle "Notes and Highlights" page. First, they changed the URL to this page, and secondly, they completely changed the structure of the page. Previously, it was possible to figure out which URL was serving up the JSON which drove the highlights, and after logging in, simply fetch highlights via that URL. As best I can tell, that has also changed. Now, the URL which serves up the highlights data returns HTML. As a result, I made some significant changes. In the interest of [overcoming my obsession with stringly-typed Ruby](http://confreaks.tv/videos/rubyconf2014-overcoming-our-obsession-with-stringly-typed-ruby), I've introduced `Book` and `Highlight` classes. Highlights are fetched through the `Book` class, via the Mechanize agent. Additionally, highlights are losing the following information: ``` "customerId" "embeddedId" "endLocation" "howLongAgo" "startLocation" "timestamp" ``` Unfortunately, it became too difficult at this time to figure out how to get that information from the "new" endpoint. I welcome anyone taking a shot at getting that information back into the `Highlight` class. Since these are breaking changes, I will bump the version to 2.0 and cut a new release sometime soon.
- Loading branch information
Showing
11 changed files
with
298 additions
and
356 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,3 +2,5 @@ | |
| .gemspec | ||
| config/*.yml | ||
| .DS_Store | ||
| application.yml | ||
| Gemfile.lock | ||
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,18 +1,20 @@ | ||
| Gem::Specification.new do |s| | ||
| s.name = "kindle-highlights" | ||
| s.version = "1.0.2" | ||
| s.version = "2.0.0" | ||
| s.summary = "Kindle highlights" | ||
| s.description = "Until there is a Kindle API, this will suffice." | ||
| s.authors = ["Eric Farkas"] | ||
| s.email = "eric@prudentiadigital.com" | ||
| s.files = ["lib/kindle_highlights.rb", "lib/kindle_highlights/client.rb"] | ||
| s.files = `git ls-files -- lib/*`.split("\n") | ||
| s.files += ["MIT-LICENSE"] | ||
| s.homepage = "https://github.com/speric/kindle-highlights" | ||
| s.license = "MIT" | ||
|
|
||
| s.required_ruby_version = ">= 2.1.0" | ||
|
|
||
| s.add_runtime_dependency "mechanize", ">= 2.7.2" | ||
| s.add_runtime_dependency "mechanize", ">= 2.7.5" | ||
| s.add_development_dependency "rake" | ||
| s.add_development_dependency "bundler", "~> 1.3" | ||
| s.add_development_dependency "minitest", "~> 5.0" | ||
| s.add_development_dependency "activesupport" | ||
| end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,8 @@ | ||
| require 'rubygems' | ||
| require 'mechanize' | ||
| require 'json' | ||
| require 'kindle_highlights/client' | ||
| require 'active_support/core_ext/object/blank' | ||
| require 'active_support/core_ext/string/filters' | ||
|
|
||
| module KindleHighlights | ||
| KINDLE_LOGIN_PAGE = "http://kindle.amazon.com/login" | ||
| SIGNIN_FORM_IDENTIFIER = "signIn" | ||
| BATCH_SIZE = 200 | ||
| end | ||
| require_relative './kindle_highlights/client' | ||
| require_relative './kindle_highlights/book' | ||
| require_relative './kindle_highlights/highlight' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| module KindleHighlights | ||
| class Book | ||
| attr_accessor :asin, :author, :title | ||
|
|
||
| def self.from_html_elements(html_element:, mechanize_agent:) | ||
| new( | ||
| mechanize_agent: mechanize_agent, | ||
| asin: html_element.attributes["id"].value.squish, | ||
| title: html_element.children.search("h2").first.text.squish, | ||
| author: html_element.children.search("p").first.text.split(":").last.strip.squish | ||
| ) | ||
| end | ||
|
|
||
| def initialize(asin:, author:, title:, mechanize_agent: nil) | ||
| @asin = asin | ||
| @author = author | ||
| @title = title | ||
| @mechanize_agent = mechanize_agent | ||
| end | ||
|
|
||
| def to_s | ||
| "#{title} by #{author}" | ||
| end | ||
|
|
||
| def inspect | ||
| "<#{self.class}: #{inspectable_vars}>" | ||
| end | ||
|
|
||
| def highlights_from_amazon | ||
| return [] unless mechanize_agent.present? | ||
|
|
||
| @highlights ||= fetch_highlights_from_amazon | ||
| end | ||
|
|
||
| private | ||
|
|
||
| attr_reader :mechanize_agent | ||
|
|
||
| def fetch_highlights_from_amazon | ||
| mechanize_agent | ||
| .get("https://read.amazon.com/kp/notebook?captcha_verified=1&asin=#{asin}&contentLimitState=&") | ||
| .search("div#kp-notebook-annotations") | ||
| .children | ||
| .select { |child| child.name == "div" } | ||
| .select { |child| child.children.search("div.kp-notebook-highlight").first.present? } | ||
| .map { |html_elements| Highlight.from_html_elements(book: self, html_elements: html_elements) } | ||
| end | ||
|
|
||
| def inspectable_vars | ||
| instance_variables | ||
| .select { |ivar| ivar != :@mechanize_agent } | ||
| .map { |ivar| "#{ivar}=#{instance_variable_get(ivar).inspect}" } | ||
| .join(", ") | ||
| end | ||
| end | ||
| end |
Oops, something went wrong.