Skip to content

Commit

Permalink
finishes name change
Browse files Browse the repository at this point in the history
  • Loading branch information
adam-hanna committed Oct 8, 2019
1 parent 762e94f commit 3d61300
Show file tree
Hide file tree
Showing 37 changed files with 136 additions and 133 deletions.
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -126,11 +126,11 @@ kill/gopls:

.PHONY: docker/host
docker/host:
@docker build -t gossip-host -f ./docker/dockerfiles/host/Dockerfile .
@docker build -t go-libp2p-pubsub-benchmark-tools -f ./docker/dockerfiles/host/Dockerfile .

.PHONY: docker/client
docker/client:
@docker build -t gossip-client -f ./docker/dockerfiles/client/Dockerfile .
@docker build -t go-libp2p-pubsub-benchmark-tools -f ./docker/dockerfiles/client/Dockerfile .

.PHONY: help
help: Makefile
Expand Down
29 changes: 16 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
> libp2p/go-libp2p-pubsub gossip host
> libp2p/go-libp2p-pubsub benchmark tools
# gossip-host
[![Build Status](https://travis-ci.org/agencyenterprise/gossip-host.svg?branch=develop)](https://travis-ci.org/agencyenterprise/gossip-host) [![Coverage Status](https://coveralls.io/repos/github/agencyenterprise/gossip-host/badge.svg?branch=develop)](https://coveralls.io/github/agencyenterprise/gossip-host?branch=develop) [![Go Report Card](https://goreportcard.com/badge/github.com/agencyenterprise/gossip-host)](https://goreportcard.com/report/github.com/agencyenterprise/gossip-host) [![GoDoc](https://godoc.org/github.com/agencyenterprise/gossip-host?status.svg)](https://godoc.org/github.com/agencyenterprise/gossip-host)
# go-libp2p-pubsub-benchmark-tools
[![Build Status](https://travis-ci.org/agencyenterprise/go-libp2p-pubsub-benchmark-tools.svg?branch=develop)](https://travis-ci.org/agencyenterprise/go-libp2p-pubsub-benchmark-tools) [![Coverage Status](https://coveralls.io/repos/github/agencyenterprise/go-libp2p-pubsub-benchmark-tools/badge.svg?branch=develop)](https://coveralls.io/github/agencyenterprise/go-libp2p-pubsub-benchmark-tools?branch=develop) [![Go Report Card](https://goreportcard.com/badge/github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools)](https://goreportcard.com/report/github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools) [![GoDoc](https://godoc.org/github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools?status.svg)](https://godoc.org/github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools)

This module provides tools for benchmarking the [libp2p gossip pubsub protocol](https://github.com/libp2p/go-libp2p-pubsub); however, it can be easily extended to other pubsub protocols or even beyond pubsub.
This module provides tools for benchmarking the [go-libp2p pubsub protocols](https://github.com/libp2p/go-libp2p-pubsub); however, it can be easily extended to other pubsub protocols or even beyond pubsub.


## Architecture

Hosts are connected to each other via the protocols defined in the config file. Each host also runs an rpc host and can receive messages from clients. Via rpc, clients can request hosts connect and disconnect with peers, gossip a message, shutdown, and more.
Hosts are connected to each other via the protocols defined in the config file. Each host also runs an rpc host and can receive messages from clients. Via rpc, clients can request hosts connect and disconnect with peers, publish a message, shutdown, and more.


## Usage

The simplest way to use the tool is via `cmd/subnet/main.go`:

1. Spin up a subnet of hosts: `$ go run ./cmd/subnet/main.go`
2. In another terminal, pass a message into the subnet and watch the hosts start gossiping: `$ go run ./cmd/client/main.go gossip`
2. In another terminal, pass a message into the subnet and watch the hosts start publishing: `$ go run ./cmd/client/main.go publish`

If you'd like to manually spin up hosts, do the following:
1. `$ go run ./cmd/host/main.go`
2. In another terminal, spin up a second host and connect it to the first: `$ go run ./cmd/host/main.go -l /ip4/127.0.0.2/tcp/3002,/ip4/127.0.0.2/tcp/3003/ws -r :8081 -p <prev. host listen addrs>`.
* Note, the `-l` flag are the listen addresses. Notice how we've incremented standard local host from `127.0.0.1` to `127.0.0.2`. We could have also simply changed the port address.
* Further note that `-r` is the rpc listen address and needs to be different for this host than the default `:8080`.
3. In a third terminal, pass a message into the subnet and watch the hosts start gossiping: `$ go run ./cmd/client/main.go gossip`
3. In a third terminal, pass a message into the subnet and watch the hosts start publishing: `$ go run ./cmd/client/main.go publish`


## Commands
Expand Down Expand Up @@ -61,7 +61,7 @@ Usage:

Flags:
-h, --help help for analyze
--log string Log file location. Defaults to standard out.
-o, --out string Output json location. Defaults to standard out.
```

An example output is:
Expand All @@ -87,7 +87,7 @@ Usage:
Available Commands:
close-all Close all peer connections
close-peers Close connections to peers
gossip Gossip a message in the pubsub
publish Publish a message in the pubsub
help Help about any command
id Get peer ids
list-peers List connected peers
Expand Down Expand Up @@ -138,6 +138,7 @@ The host has many configuration options which can be set between a combination o
"peers": [],
"muxers": [["yamux", "/yamux/1.0.0"], ["mplex", "/mplex/6.7.0"]],
"security": "secio",
"pubsubAlgorithm": "gossip",
"omitRelay": false,
"omitConnectionManager": false,
"omitNATPortMap": false,
Expand Down Expand Up @@ -196,6 +197,7 @@ The orchestration can be configured via a json file. The default configuration l
"transports": ["tcp", "ws"],
"muxers": [["yamux", "/yamux/1.0.0"], ["mplex", "/mplex/6.7.0"]],
"security": "secio",
"pubsubAlgorithm": "gossip",
"omitRelay": false,
"omitConnectionManager": false,
"omitNATPortMap": false,
Expand All @@ -218,7 +220,7 @@ The available commands and flags are shown below.

```bash
$ go run ./cmd/subnet/main.go --help
Start a subnet of interconnected gossipsub hosts
Start a subnet of interconnected libp2p pubsub hosts

Usage:
start [flags]
Expand Down Expand Up @@ -247,6 +249,7 @@ The subnet can be configured via a json file. The default configuration location
"transports": ["tcp", "ws"],
"muxers": [["yamux", "/yamux/1.0.0"], ["mplex", "/mplex/6.7.0"]],
"security": "secio",
"pubsubAlgorithm": "gossip",
"omitRelay": false,
"omitConnectionManager": false,
"omitNATPortMap": false,
Expand Down Expand Up @@ -290,8 +293,8 @@ syntax = "proto3";
package pb;
option java_multiple_files = true;
option java_package = "io.grpc.gossipsub.benchmark";
option java_outer_classname = "GossipsubBenchmark";
option java_package = "io.grpc.pubsub.benchmark";
option java_outer_classname = "PubsubBenchmark";
import "google/protobuf/empty.proto";
Expand Down Expand Up @@ -364,7 +367,7 @@ service Publisher {

[**MIT**](LICENSE).

[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fagencyenterprise%2Fgossip-host.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fagencyenterprise%2Fgossip-host?ref=badge_large)
[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fagencyenterprise%2Fgo-libp2p-pubsub-benchmark-tools.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fagencyenterprise%2Fgo-libp2p-pubsub-benchmark-tools?ref=badge_large)

```
MIT License
Expand Down
4 changes: 2 additions & 2 deletions cmd/analysis/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ import (
"encoding/json"
"os"

"github.com/agencyenterprise/gossip-host/pkg/analysis"
"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/analysis"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"

"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
Expand Down
14 changes: 7 additions & 7 deletions cmd/client/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ package main
import (
"os"

"github.com/agencyenterprise/gossip-host/pkg/client"
"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/client"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"

"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
Expand All @@ -20,16 +20,16 @@ func setup() (*cobra.Command, error) {
rootCmd := &cobra.Command{Use: "client"}

publishCMD := &cobra.Command{
Use: "gossip",
Short: "Gossip a message in the pubsub",
Long: "Request that the passed hosts gossip the passed message",
Use: "publish",
Short: "Publish a message in the pubsub",
Long: "Request that the passed hosts publish the passed message",
Run: func(cmd *cobra.Command, args []string) {
if err := logger.Set(logger.ContextHook{}, loggerLoc, false); err != nil {
logrus.Fatalf("err initiating logger:\n%v", err)
}

logger.Infof("sending message to peers for gossip")
if err := client.Gossip(nil, msgLoc, peers, size, timeout); err != nil {
logger.Infof("sending message to peers for publish")
if err := client.Publish(nil, msgLoc, peers, size, timeout); err != nil {
logger.Fatalf("err sending messages\n%v", err)
}
},
Expand Down
8 changes: 4 additions & 4 deletions cmd/host/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import (
"os/signal"
"syscall"

"github.com/agencyenterprise/gossip-host/pkg/host"
"github.com/agencyenterprise/gossip-host/pkg/host/config"
"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/host"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/host/config"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"

"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
Expand All @@ -22,7 +22,7 @@ func setup() *cobra.Command {
rootCmd := &cobra.Command{
Use: "start",
Short: "Start node",
Long: `Starts the gossip pub/sub node`,
Long: `Starts the go-libp2p pub/sub host`,
RunE: func(cmd *cobra.Command, args []string) error {
if err := logger.Set(logger.ContextHook{}, loggerLoc, false); err != nil {
logrus.Errorf("err initiating logger:\n%v", err)
Expand Down
12 changes: 6 additions & 6 deletions cmd/orchestrate/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ import (
"syscall"
"time"

"github.com/agencyenterprise/gossip-host/pkg/client"
"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/gossip-host/pkg/orchestrate/config"
"github.com/agencyenterprise/gossip-host/pkg/subnet"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/client"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/orchestrate/config"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/subnet"

"github.com/google/uuid"
"github.com/sirupsen/logrus"
Expand Down Expand Up @@ -143,8 +143,8 @@ func setup() *cobra.Command {
peerIdx := randBetween(0, len(peers)-1)
peer := peers[peerIdx]

logger.Infof("sending message to %s for gossip", peer)
if err := client.Gossip([]byte(id.String()), c.Orchestra.MessageLocation, peer, c.Orchestra.MessageByteSize, c.Orchestra.ClientTimeoutSeconds); err != nil {
logger.Infof("sending message to %s for publish", peer)
if err := client.Publish([]byte(id.String()), c.Orchestra.MessageLocation, peer, c.Orchestra.MessageByteSize, c.Orchestra.ClientTimeoutSeconds); err != nil {
logger.Fatalf("err sending messages\n%v", err)
e <- err
}
Expand Down
8 changes: 4 additions & 4 deletions cmd/subnet/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ import (
"os/signal"
"syscall"

"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/gossip-host/pkg/subnet"
"github.com/agencyenterprise/gossip-host/pkg/subnet/config"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/subnet"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/subnet/config"

"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
Expand All @@ -23,7 +23,7 @@ func setup() *cobra.Command {
rootCmd := &cobra.Command{
Use: "start",
Short: "Start subnet",
Long: `Start a subnet of interconnected gossipsub hosts`,
Long: `Start a subnet of interconnected libp2p pubsub hosts`,
RunE: func(cmd *cobra.Command, args []string) error {
if err := logger.Set(logger.ContextHook{}, loggerLoc, false); err != nil {
logrus.Errorf("err initiating logger:\n%v", err)
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
module github.com/agencyenterprise/gossip-host
module github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools

go 1.13

Expand Down
4 changes: 2 additions & 2 deletions pkg/analysis/analysis.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package analysis

import (
"github.com/agencyenterprise/gossip-host/pkg/analysis/types"
"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/analysis/types"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"
)

// Analyze parses a log file and returns performance metrics
Expand Down
2 changes: 1 addition & 1 deletion pkg/analysis/sorter/sorter.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package sorter
import (
"sort"

"github.com/agencyenterprise/gossip-host/pkg/analysis/types"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/analysis/types"
)

// By is the type of a "less" function that defines the ordering of its Planet arguments.
Expand Down
2 changes: 1 addition & 1 deletion pkg/analysis/types/types.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package types

import "github.com/agencyenterprise/gossip-host/pkg/cerr"
import "github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/cerr"

const (
// ErrImproperlyFormattedLogLine is returned when the log line is not in the expected format
Expand Down
6 changes: 3 additions & 3 deletions pkg/analysis/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ import (
"strconv"
"strings"

"github.com/agencyenterprise/gossip-host/pkg/analysis/sorter"
"github.com/agencyenterprise/gossip-host/pkg/analysis/types"
"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/analysis/sorter"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/analysis/types"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"
)

func loadLogFile(logLoc string) (*bufio.Scanner, func() error, error) {
Expand Down
12 changes: 6 additions & 6 deletions pkg/client/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ import (
"errors"
"time"

"github.com/agencyenterprise/gossip-host/pkg/logger"
pb "github.com/agencyenterprise/gossip-host/pkg/pb/publisher"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"
pb "github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/pb/publisher"
"github.com/golang/protobuf/ptypes/empty"

grpc "google.golang.org/grpc"
)

// Gossip requests the passed peers to gossip the passed message
func Gossip(msgID []byte, msgLoc, peers string, size uint, timeout int) error {
// Publish requests the passed peers to publish the passed message
func Publish(msgID []byte, msgLoc, peers string, size uint, timeout int) error {
msg, err := parseMessageFile(msgLoc)
if err != nil || msg == nil {
logger.Errorf("err parsing message file:\n%v", err)
Expand Down Expand Up @@ -53,7 +53,7 @@ func Gossip(msgID []byte, msgLoc, peers string, size uint, timeout int) error {

r, err := c.PublishMessage(ctx, msg)
if err != nil {
logger.Errorf("could not gossip message to %s:\n %v", peer, err)
logger.Errorf("could not publish message to %s:\n %v", peer, err)
failed = true
conn.Close()
cancel()
Expand All @@ -70,7 +70,7 @@ func Gossip(msgID []byte, msgLoc, peers string, size uint, timeout int) error {
}

if failed {
return errors.New("gossip failed")
return errors.New("publish failed")
}

return nil
Expand Down
4 changes: 2 additions & 2 deletions pkg/client/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import (
"io/ioutil"
"strings"

"github.com/agencyenterprise/gossip-host/pkg/logger"
pb "github.com/agencyenterprise/gossip-host/pkg/pb/publisher"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"
pb "github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/pb/publisher"
)

func parseMessageFile(loc string) (*pb.Message, error) {
Expand Down
4 changes: 2 additions & 2 deletions pkg/grpc/host/host.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import (
"net"
"time"

"github.com/agencyenterprise/gossip-host/pkg/logger"
pb "github.com/agencyenterprise/gossip-host/pkg/pb/publisher"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"
pb "github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/pb/publisher"

"github.com/golang/protobuf/ptypes/empty"
ipfsaddr "github.com/ipfs/go-ipfs-addr"
Expand Down
4 changes: 2 additions & 2 deletions pkg/grpc/host/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ import (
"encoding/binary"
"time"

"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"

pb "github.com/agencyenterprise/gossip-host/pkg/pb/publisher"
pb "github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/pb/publisher"
"github.com/davecgh/go-spew/spew"
peer "github.com/libp2p/go-libp2p-peer"
pubsub "github.com/libp2p/go-libp2p-pubsub"
Expand Down
2 changes: 1 addition & 1 deletion pkg/host/config/config.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package config

import (
"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"
)

// Load reads the passed config file location and parses it into a config struct.
Expand Down
2 changes: 1 addition & 1 deletion pkg/host/config/types.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package config

import (
"github.com/agencyenterprise/gossip-host/pkg/cerr"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/cerr"

lcrypto "github.com/libp2p/go-libp2p-core/crypto"
)
Expand Down
2 changes: 1 addition & 1 deletion pkg/host/config/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (
"path/filepath"
"strings"

"github.com/agencyenterprise/gossip-host/pkg/logger"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"

"github.com/gobuffalo/packr/v2"
lcrypto "github.com/libp2p/go-libp2p-core/crypto"
Expand Down
6 changes: 3 additions & 3 deletions pkg/host/host.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ import (
"strings"
"time"

rpcHost "github.com/agencyenterprise/gossip-host/pkg/grpc/host"
"github.com/agencyenterprise/gossip-host/pkg/host/config"
"github.com/agencyenterprise/gossip-host/pkg/logger"
rpcHost "github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/grpc/host"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/host/config"
"github.com/agencyenterprise/go-libp2p-pubsub-benchmark-tools/pkg/logger"

ipfsaddr "github.com/ipfs/go-ipfs-addr"
"github.com/libp2p/go-libp2p"
Expand Down
Loading

0 comments on commit 3d61300

Please sign in to comment.