Number of cycles in the list of values, which are a mixture of positive and negative in Spark and Scala

advertisements

Have an RDD with List of values, which are mix of positives and negatives. Need to compute number of cycles from this data.

For example,

val range = List(sampleRange(2020,2030,2040,2050,-1000,-1010,-1020,Starting point,-1030,2040,-1020,2050,2040,2020,end point,-1060,-1030,-1010)

the interval between each value in above list is 1 second. ie., 2020 and 2030 are recorded in 1 second interval and so on.

how many times it turns from negative to positive and stays positive for >= 2 seconds.
If >= 2 seconds it is a cycle.

Number of cycles: Logic
Example 1: List(1,2,3,4,5,6,-15,-66)
No. of cycles is 1.
Reason: As we move from 1st element of list to 6th element, we had 5 intervals which means 5 seconds. So one cycle. As we move to 6th element of list, it is a negative value. So we start counting from 6th element and move to 7th element. The negative values are only 2 and interval is only 1. So not counted as cycle.
Example 2:
List(11,22,33,-25,-36,-43,20,25,28)
No. of cycles is 3.
Reason: As we move from 1st element of list to 3rd element, we had 2 intervals which means 2 seconds. So one cycle As we move to 4th element of list, it is a negative value. So we start counting from 4th element and move to 5th, 6th element. we had 2 intervals which means 2 seconds. So one cycle As we move to 7th element of list, it is a positive value. So we start counting from 7th element and move to 8th, 9th element. we had 2 intervals which means 2 seconds. So one cycle.

range is a RDD in the use case. It looks like
scala> range
range: Seq[com.Range] = List(XtreamRange(858,890,899,920,StartEngage,-758,-790,-890,-720,920,940,950))


You can encode this "how many times it turns from negative to positive and stays positive for >= 2 seconds. If >= 2 seconds it is a cycle." pretty much directly into a pattern match with a guard. The expression if(h < 0 && ht > 0 && hht > 0) checks for a cycle and adds one to the result then continues with the rest of the list.

def countCycles(xs: List[Int]): Int = xs match {
 case Nil => 0
 case h::ht::hht::t if(h < 0 && ht > 0 && hht > 0) => 1 + countCycles(t)
 case h::t => countCycles(t)
}

scala> countCycles(range)
res7: Int = 1