Skip to content

Commit

Permalink
Move to a filters directory
Browse files Browse the repository at this point in the history
  • Loading branch information
clipperhouse committed Apr 8, 2020
1 parent c81c389 commit 701df22
Show file tree
Hide file tree
Showing 36 changed files with 28 additions and 28 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ import (
"strings"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/stackoverflow"
"github.com/clipperhouse/jargon/filters/stackoverflow"
)

text := `Let’s talk about Ruby on Rails and ASPNET MVC.`
Expand All @@ -72,17 +72,17 @@ if err := stream.Err(); err != nil {

Canonical terms (lemmas) are looked up in token filters. Several are available:

[Stack Overflow technology tags](https://pkg.go.dev/github.com/clipperhouse/jargon/stackoverflow)
[Stack Overflow technology tags](https://pkg.go.dev/github.com/clipperhouse/jargon/filters/stackoverflow)
- `Ruby on Rails → ruby-on-rails`
- `ObjC → objective-c`

[Contractions](https://pkg.go.dev/github.com/clipperhouse/jargon/contractions)
[Contractions](https://pkg.go.dev/github.com/clipperhouse/jargon/filters/contractions)
- `Couldn’t → Could not`

[ASCII fold](https://pkg.go.dev/github.com/clipperhouse/jargon/ascii)
[ASCII fold](https://pkg.go.dev/github.com/clipperhouse/jargon/filters/ascii)
- `café → cafe`

[Stem](https://pkg.go.dev/github.com/clipperhouse/jargon/stemmer)
[Stem](https://pkg.go.dev/github.com/clipperhouse/jargon/filters/stemmer)
- `Manager|management|manages → manag`

To implement your own, see the [Filter type](https://godoc.org/github.com/clipperhouse/jargon/#Filter).
Expand Down
10 changes: 5 additions & 5 deletions cmd/jargon/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ import (
"strings"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/ascii"
"github.com/clipperhouse/jargon/contractions"
"github.com/clipperhouse/jargon/stackoverflow"
"github.com/clipperhouse/jargon/stemmer"
"github.com/clipperhouse/jargon/filters/ascii"
"github.com/clipperhouse/jargon/filters/contractions"
"github.com/clipperhouse/jargon/filters/stackoverflow"
"github.com/clipperhouse/jargon/filters/stemmer"
"github.com/spf13/afero"
)

Expand Down Expand Up @@ -136,7 +136,7 @@ func setInput(c *config, mode os.FileMode, filein string) error {
c.Filein = file
}

c.Pipedin = (mode & os.ModeCharDevice) == 0 // https://stackoverflow.com/a/43947435/70613
c.Pipedin = (mode & os.ModeCharDevice) == 0 // https://filters/stackoverflow.com/a/43947435/70613

// If no input, display usage
input := c.Pipedin || c.Filein != nil
Expand Down
10 changes: 5 additions & 5 deletions cmd/jargon/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ import (
"testing"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/ascii"
"github.com/clipperhouse/jargon/contractions"
"github.com/clipperhouse/jargon/stackoverflow"
"github.com/clipperhouse/jargon/stemmer"
"github.com/clipperhouse/jargon/filters/ascii"
"github.com/clipperhouse/jargon/filters/contractions"
"github.com/clipperhouse/jargon/filters/stackoverflow"
"github.com/clipperhouse/jargon/filters/stemmer"
"github.com/spf13/afero"
)

Expand Down Expand Up @@ -172,7 +172,7 @@ func TestFilters(t *testing.T) {
}
if len(c.Filters) == len(test.filters) {
for i := range test.filters {
// https://stackoverflow.com/a/9644797
// https://filters/stackoverflow.com/a/9644797
// This is not a good test, but perhaps better than nothing
// E.g. the pointers to different stemmers are all the same
expected := reflect.ValueOf(test.filters[i]).Pointer()
Expand Down
2 changes: 1 addition & 1 deletion example_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"strings"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/stackoverflow"
"github.com/clipperhouse/jargon/filters/stackoverflow"
)

func ExampleTokenize() {
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
6 changes: 3 additions & 3 deletions contractions/README.md → filters/contractions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import (
"fmt"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/contractions"
"github.com/clipperhouse/jargon/filters/contractions"
)

var lem = jargon.NewLemmatizer(contractions.Expander)
Expand All @@ -52,6 +52,6 @@ func main() {

### Implementation

The [Lookup method](https://github.com/clipperhouse/jargon/blob/master/contractions/filter.go#L7) satisfies the [jargon.TokenFilter interface](https://github.com/clipperhouse/jargon/blob/master/filter.go).
The [Lookup method](https://github.com/clipperhouse/jargon/blob/master/filters/contractions/filter.go#L7) satisfies the [jargon.TokenFilter interface](https://github.com/clipperhouse/jargon/blob/master/filter.go).

Here is the [base list of contractions](https://github.com/clipperhouse/jargon/blob/master/contractions/generator.go#L101). Variations (case, apostrophes) are code-generated.
Here is the [base list of contractions](https://github.com/clipperhouse/jargon/blob/master/filters/contractions/generator.go#L101). Variations (case, apostrophes) are code-generated.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"testing"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/contractions"
"github.com/clipperhouse/jargon/filters/contractions"
)

func TestContractions(t *testing.T) {
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion nba/filter.go → filters/nba/filter.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package nba

import "github.com/clipperhouse/jargon/synonyms"
import "github.com/clipperhouse/jargon/filters/synonyms"

//go:generate go run generate/main.go

Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion nba/generate/main.go → filters/nba/generate/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"text/template"

"github.com/PuerkitoBio/goquery"
"github.com/clipperhouse/jargon/ascii"
"github.com/clipperhouse/jargon/filters/ascii"
)

func main() {
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
package stackoverflow

import (
"github.com/clipperhouse/jargon/synonyms"
"github.com/clipperhouse/jargon/filters/synonyms"
)

//go:generate go run generate/main.go
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"testing"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/stopwords"
"github.com/clipperhouse/jargon/filters/stopwords"
)

func TestCaseSensitive(t *testing.T) {
Expand Down
2 changes: 1 addition & 1 deletion synonyms/filter.go → filters/synonyms/filter.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"sync"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/synonyms/trie"
"github.com/clipperhouse/jargon/filters/synonyms/trie"
)

// Filter is the data structure of a synonyms filter. Use NewFilter to create.
Expand Down
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion twitter/filter.go → filters/twitter/filter.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"unicode"
"unicode/utf8"

"github.com/clipperhouse/jargon/sigil"
"github.com/clipperhouse/jargon/filters/sigil"
)

// Handles will identify Twitter-style handles, combining the @ and name into a single token
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion tokenstream_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"strings"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/stackoverflow"
"github.com/clipperhouse/jargon/filters/stackoverflow"
)

func ExampleTokenStream_Scan() {
Expand Down
2 changes: 1 addition & 1 deletion web/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"strings"

"github.com/clipperhouse/jargon"
"github.com/clipperhouse/jargon/stackoverflow"
"github.com/clipperhouse/jargon/filters/stackoverflow"
)

func main() {
Expand Down

0 comments on commit 701df22

Please sign in to comment.