Machine Learning/Kaggle Social Network Contest/Features - Revision history

Jjhale: /* Joe's attempt */

2010-11-25T23:50:55Z

Joe's attempt

← Older revision		Revision as of 16:50, 25 November 2010
Line 36:		Line 36:

	For an edge from node s to node t I will calculate:		For an edge from node s to node t I will calculate:
			# is there a directed edge from t to s?
	# the in-degree of s		# the in-degree of s
	# the out-degree of s		# the out-degree of s

Jjhale: /* Joe's attempt */

2010-11-25T23:44:58Z

Joe's attempt

← Older revision		Revision as of 16:44, 25 November 2010
Line 57:		Line 57:

	where		where
	* RLD<sub>x</sub>(n) is 1 / log(the x-degree of node n), where -1 = in, 1 = out and 0 = any. (RLD = reciprocal log of degree )		* RLD<sub>x</sub>(n) is 1 / log(0.1 + the x-degree of node n), where -1 = in, 1 = out and 0 = any. (RLD = reciprocal log of degree )
			** note that I add 0.1 so that nodes with degree 1 have a score of 1/log(1.1) = 10.49 rather than1/log(1) which is a divide by zero
			** logs are taken to base e

	I define N<sub>x</sub><sup>h</sup>(n) to be the nodes reachable from ''n'' in ''h'' hops along either any edge (x = 0), edges from t towards s (x = -1) or edges from s towards t (x = 1).		I define N<sub>x</sub><sup>h</sup>(n) to be the nodes reachable from ''n'' in ''h'' hops along either any edge (x = 0), edges from t towards s (x = -1) or edges from s towards t (x = 1).

Jjhale: Adding my approah

2010-11-25T23:28:55Z

Adding my approah

← Older revision		Revision as of 16:28, 25 November 2010
Line 31:		Line 31:

	The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future		The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future

			== Joe's attempt ==
			I'm planning on collecting features based on an edge. Then sample the features over existing and randomly created edges and fit a logistic regression model to it.

			For an edge from node s to node t I will calculate:
			# the in-degree of s
			# the out-degree of s
			# the in-degree of t
			# the out-degree of t
			# RLD<sub>-1</sub>(s)
			# RLD<sub>1</sub>(s)
			# RLD<sub>0</sub>(s)
			# RLD<sub>-1</sub>(t)
			# RLD<sub>1</sub>(t)
			# RLD<sub>0</sub>(t)
			#AA<sub>0</sub><sup>1</sup>(s,t)
			# AA<sub>0</sub><sup>1.5</sup>(s,t)
			# AA<sub>0</sub><sup>2</sup>(s,t)
			# AA<sub>-1</sub><sup>1</sup>(s,t)
			# AA<sub>-1</sub><sup>1.5</sup>(s,t)
			# AA<sub>-1</sub><sup>2</sup>(s,t)
			# AA<sub>1</sub><sup>1</sup>(s,t)
			# AA<sub>1</sub><sup>1.5</sup>(s,t)
			# AA<sub>1</sub><sup>2</sup>(s,t)

			where
			* RLD<sub>x</sub>(n) is 1 / log(the x-degree of node n), where -1 = in, 1 = out and 0 = any. (RLD = reciprocal log of degree )

			I define N<sub>x</sub><sup>h</sup>(n) to be the nodes reachable from ''n'' in ''h'' hops along either any edge (x = 0), edges from t towards s (x = -1) or edges from s towards t (x = 1).

			I define C<sub>x</sub><sup>h</sup>(s,t) as the set of common neighbours of s and t a distance of h hops from s and t, excluding nodes in a closer common neighbourhood ie
			* C<sub>x</sub><sup>h</sup>(s,t) = (N<sub>x</sub><sup>h</sup>(s) ∩ N<sub>-x</sub><sup>h</sup>(t)) \ ∪<sub>h' < h </sub>(N<sub>x</sub><sup>h'</sup>(s) ∩ N<sub>-x</sub><sup>h'</sup>(t))
			** h = 1.5 corresponds to nodes which are one hop from either s or t and two hops from either t or s
			* The sets C<sub>x</sub><sup>h</sup>(s,t) are distinct for different h.
			* It is directional, ie sometimes C<sub>x</sub><sup>h</sup>(s,t)≠C<sub>x</sub><sup>h</sup>(t,s)
			* AA is the Adamic-Adar score calculated over different common neighbourhoods.
			** the subscript 0, -1, 1 referes to neighbours reachable be following any, in or out node respectively
			** the superscript 1, 1.5 and 2 refer to the the number of hops from a focal node the neighbour is.
			* AA<sub>x</sub><sup>h</sup>(s,t) = sum<sub>n ∈ C<sub>x</sub><sup>h</sup>(s,t)</sub> RLD<sub>0</sub>(n)

Jjhale: /* Possible Features */

2010-11-23T04:05:53Z

Possible Features

← Older revision		Revision as of 21:05, 22 November 2010
Line 25:		Line 25:
	*** see [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.1370&rep=rep1&type=pdf original paper]		*** see [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.1370&rep=rep1&type=pdf original paper]
	*** R igraph: [http://cneurocvs.rmki.kfki.hu/igraph/doc/R/similarity.html similarity.invlogweighted]		*** R igraph: [http://cneurocvs.rmki.kfki.hu/igraph/doc/R/similarity.html similarity.invlogweighted]

			* Clustering
			** membership of the same strongly connected cluster
			*** using [http://cneurocvs.rmki.kfki.hu/igraph/doc/R/clusters.html igraph clusters]

	The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future		The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future

Jjhale: /* Possible Features */

2010-11-23T02:32:47Z

Possible Features

← Older revision		Revision as of 19:32, 22 November 2010
Line 24:		Line 24:
	** Adamic-Adar score		** Adamic-Adar score
	*** see [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.1370&rep=rep1&type=pdf original paper]		*** see [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.1370&rep=rep1&type=pdf original paper]
			*** R igraph: [http://cneurocvs.rmki.kfki.hu/igraph/doc/R/similarity.html similarity.invlogweighted]

	The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future		The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future

Jared: /* Possible Features */

2010-11-20T11:25:57Z

Possible Features

← Older revision		Revision as of 04:25, 20 November 2010
Line 15:		Line 15:
	**shortest distance nodeid to nodetofollowid		**shortest distance nodeid to nodetofollowid
	**density? (<strike>median path length</strike>)		**density? (<strike>median path length</strike>)
	**is nodetofollowid following nodeid?		**does reverse edge exist? (aka is nodetofollowid following nodeid?)
	**number of common friends		**number of common friends
	**indegrees & outdegrees of nodetofollowid		**indegrees & outdegrees of nodetofollowid

Jared: /* Possible Features */

2010-11-20T05:54:50Z

Possible Features

← Older revision		Revision as of 22:54, 19 November 2010
Line 3:		Line 3:

	== Possible Features ==		== Possible Features ==
	*nodeid		*Node Features
	*~~nodetofollowid~~		**nodeid
	*~~median path length~~		**outdegree
	*~~shortest distance from nodeid to nodetofollowid~~		**indegree
	*inbound edges		**local clustering coefficient
	*outbound edges		**reciprocation of inbound probability (num of edges returned / num of inbound edges)
	*~~clustering coefficient~~		**reciprocation of outbound probability (num of edges returned / num of outbound edges)
	*reciprocation probability (num of edges returned / num of outbound edges)

	~~The response variable~~ is ~~the probability that the~~ nodeid to nodetofollowid ~~edge will be created in the future~~		*Edge Features
			**nodetofollowid
			**shortest distance nodeid to nodetofollowid
			**density? (<strike>median path length</strike>)
			**is nodetofollowid following nodeid?
			**number of common friends
			**indegrees & outdegrees of nodetofollowid

	~~From the Backstrom and Leskovec, for a node s and a potential target c~~
	* Network features		* Network features
	** unweighted random walk score		** unweighted random walk score
			** global clustering coefficient
	** Adamic-Adar score		** Adamic-Adar score
	*** see [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.1370&rep=rep1&type=pdf original paper]		*** see [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.1370&rep=rep1&type=pdf original paper]
	** number of common friends
	** indegrees and outdegrees of s		The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future
	*** the indegree is the ~~number of edges coming into node s~~
	~~***~~ the ~~outdegree is~~ the ~~number of edges leaving node s~~
	** indegrees and outdegrees of c

Jjhale: /* Possible Features */

2010-11-20T05:30:14Z

Possible Features

← Older revision		Revision as of 22:30, 19 November 2010
Line 18:		Line 18:
	** unweighted random walk score		** unweighted random walk score
	** Adamic-Adar score		** Adamic-Adar score
			*** see [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.1370&rep=rep1&type=pdf original paper]
	** number of common friends		** number of common friends
	** indegrees and outdegrees of s		** indegrees and outdegrees of s

Jjhale at 05:17, 20 November 2010

2010-11-20T05:17:08Z

@@ Line 1: / Line 1: @@
 == Possible Features ==
 *nodeid
@@ Line 10: / Line 13: @@
 The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future

Jjhale: Created page with '== Possible Features == nodeid nodetofollowid median path length shortest distance from nodeid to nodetofollowid inbound edges outbound edges clustering coefficient recip…'

2010-11-19T07:12:13Z

Created page with '== Possible Features == *nodeid *nodetofollowid *median path length *shortest distance from nodeid to nodetofollowid *inbound edges *outbound edges *clustering coefficient *recip…'

New page

== Possible Features ==
*nodeid
*nodetofollowid
*median path length
*shortest distance from nodeid to nodetofollowid
*inbound edges
*outbound edges
*clustering coefficient
*reciprocation probability (num of edges returned / num of outbound edges)

The response variable is the probability that the nodeid to nodetofollowid edge will be created in the future

Machine Learning/Kaggle Social Network Contest/Features - Revision history

Jjhale: /* Joe's attempt */

Jjhale: /* Joe's attempt */

Jjhale: Adding my approah

Jjhale: /* Possible Features */

Jjhale: /* Possible Features */

Jared: /* Possible Features */

Jared: /* Possible Features */

Jjhale: /* Possible Features */

Jjhale at 05:17, 20 November 2010

Jjhale: Created page with '== Possible Features == *nodeid *nodetofollowid *median path length *shortest distance from nodeid to nodetofollowid *inbound edges *outbound edges *clustering coefficient *recip…'

Jjhale: Created page with '== Possible Features == nodeid nodetofollowid median path length shortest distance from nodeid to nodetofollowid inbound edges outbound edges clustering coefficient recip…'