Graph Databases: January 2017

Issue: A Neo4j 3.1 embedded graph database does not expose the jetty based http browser (7474) by default

Solution: Wrap the embedded graph database with a NeoServer
It turns out with a bit of help from the following post we can still bring up a wrapping NeoServer around our embedded mode Neo4j 3.1.0 server
http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper

We override the following function

@Override

protected GraphDatabaseBuilder.DatabaseCreator createDatabaseCreator(

final File storeDir, final GraphDatabaseFactoryState state) {

return new GraphDatabaseBuilder.DatabaseCreator() {

@Override

public GraphDatabaseService newDatabase( final Map<String, String> config ) {

EnterpriseBootstrapper neoServer = new EnterpriseBootstrapper();

GraphDatabaseService graph = null;

// convert all config (spring, conf, code) to vararg Pairs

List<Pair<String, String>> pairs = new ArrayList<>();

for(Entry<String, String> entry : config.entrySet()) {

pairs.add(Pair.of(entry.getKey(), entry.getValue()));

}

Pair<String, String> pairArray[] = new Pair[pairs.size()];

// will resolve to /dir/data/databases/graph.db

int state = neoServer.start(storeDir, Optional.empty(), pairs.toArray(pairArray));

// state is 0 for success, 1 will mean a null server

if(state > 0) {

log.error("return state of NeoServer.start(); is 1 - no GraphDatabase available - check config settings");

}

// will be null if state == 1

NeoServer server = neoServer.getServer();

if(null != server) {

Database database = server.getDatabase();

if(null != database) {

graph = database.getGraph();

// set the paxos HA listener only when dbms.mode=HA

// Note: initial TO_MASTER callback during server.start above is missed

if(graph instanceof HighlyAvailableGraphDatabase) {

HighlyAvailableGraphDatabase haGraph = (HighlyAvailableGraphDatabase) graph;

haMonitor.setDb(haGraph);

HighAvailabilityMemberStateMachine memberStateMachine =

(haGraph).getDependencyResolver()

.resolveDependency(HighAvailabilityMemberStateMachine.class);

if ( memberStateMachine != null ) {

memberStateMachine.addHighAvailabilityMemberListener(haMonitor);

System.out.println("register: " + haMonitor);

// rethrow isMaster callback from start

//haMonitor.getMasterListenerManager().masterChanged(haGraph.isMaster());

System.out.println("isMaster: " + haGraph.isMaster());

}

} else {

log.error("database null : check your http configuration settings");

}

} else {

log.error("server null : check your http configuration settings");

}

return graph;

} };

}

Using the following spring configuration

<context:annotation-config />

<context:spring-configured />

<mvc:annotation-driven />

<context:component-scan base-package="org.obrienlabs.nbi.graph.service" />

<util:map id="config">

</util:map>

<constructor-arg ref="haMonitor" />

</bean>

<bean id="graphDbBuilder"

factory-bean="graphDbFactory"

factory-method="newEmbeddedDatabaseBuilder">

<constructor-arg value="/ec2-user"/>

</bean>

<bean id="graphDbBuilderFinal"

factory-bean="graphDbBuilder"

factory-method="setConfig">

<constructor-arg ref="config"/>

</bean>

<bean id="graphDatabaseService"

factory-bean="graphDbBuilderFinal"

factory-method="newGraphDatabase"

destroy-method="shutdown" />

Details:

Run the following code in
https://github.com/obrienlabs/nbi-neo4j-embedded-aws-war

Neo4j 3.1.0 HA (paxos) Embedded mode HighlyAvailableGraphDatabase wrapped with an EnterpriseBootstrapper NeoServer

In this code we get everything (except causal cluster using the raft protocol).

With a few modifications I was able to wrap a HighlyAvailableGraphDatabase with a EnterpriseBootstrapper.

There are some non-fatal exceptions around JMX reporting in debug.log likely related to my 8.0.28 tomcat version but the graph db running embedded in Tomcat is OK.

2016-12-21 16:20:00.574+0000 INFO Bolt enabled on 0.0.0.0:7688.
2016-12-21 16:20:09.554+0000 INFO Attempting to join cluster of [127.0.0.1:5001]
2016-12-21 16:20:11.566+0000 INFO Creating new cluster with name [neo4j.ha]...
2016-12-21 16:20:11.607+0000 INFO Instance 1 (this server) entered the cluster
2016-12-21 16:20:12.164+0000 INFO I am 1, moving to master
2016-12-21 16:20:12.293+0000 INFO Instance 1 (this server) was elected as coordinator
2016-12-21 16:20:12.462+0000 INFO I am 1, successfully moved to master
2016-12-21 16:20:12.513+0000 INFO Instance 1 (this server) is available as master at ha://127.0.0.1:6001?serverId=1 with StoreId{creationTime=1482199697648, randomId=7800059877674392627, storeVersion=15531981201765894, upgradeTime=1482199697648, upgradeId=1}
2016-12-21 16:20:14.495+0000 INFO Database available for write transactions
2016-12-21 16:20:31.917+0000 INFO Mounted REST API at: /db/manage
2016-12-21 16:20:53.264+0000 INFO Remote interface available at http://localhost:7575/
register: org.obrienlabs.nbi.graph.service.HaMonitor@1c0f80c9

Source:
https://github.com/obrienlabs/nbi-neo4j-embedded-aws-war

Diagnostics:

7575 http and 7688 bolt ports

```

obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7575

tcp46 0 0 *.7575 *.* LISTEN 131072 131072 49013 0

tcp4 0 0 127.0.0.1.7575 127.0.0.1.60685 TIME_WAIT 407296 146988 49013 0

tcp6 0 0 ::1.7575 ::1.60699 TIME_WAIT 407284 146808 49013 0

tcp6 0 0 ::1.7575 ::1.60700 TIME_WAIT 407284 146808 49013 0

obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7688

tcp6 0 0 ::1.7688 ::1.60704 ESTABLISHED 406582 146808 49013 0

tcp6 0 0 ::1.60704 ::1.7688 ESTABLISHED 398196 146808 48165 0

tcp6 0 0 ::1.7688 ::1.60702 ESTABLISHED 406570 146808 49013 0