Class MapReducerIgniteAffinityCall<X>

  • All Implemented Interfaces:
    Serializable, Mappable<X>

    public class MapReducerIgniteAffinityCall<X>
    extends MapReducer<X>
    Main class of oshdb's "functional programming" API.

    It accepts a list of filters, transformation `map` functions a produces a result when calling the `reduce` method (or one of its shorthand versions like `sum`, `count`, etc.).

    You can set a list of filters that are applied on the raw OSM data, for example you can filter:

    • geometrically by an area of interest (bbox or polygon)
    • by osm tags (key only or key/value)
    • by OSM type
    • custom filter callback

    Depending on the used data "view", the MapReducer produces either "snapshots" or evaluated all modifications ("contributions") of the matching raw OSM data.

    These data can then be transformed arbitrarily by user defined `map` functions (which take one of these entity snapshots or modifications as input an produce an arbitrary output) or `flatMap` functions (which can return an arbitrary number of results per entity snapshot/contribution). It is possible to chain together any number of transformation functions.

    Finally, one can either use one of the pre-defined result-generating functions (e.g. `sum`, `count`, `average`, `uniq`), or specify a custom `reduce` procedure.

    If one wants to get results that are aggregated by timestamp (or some other index), one can use the `aggregateByTimestamp` or `aggregateBy` functionality that automatically handles the grouping of the output data.

    For more complex analyses, it is also possible to enable the grouping of the input data by the respective OSM ID. This can be used to view at the whole history of entities at once.

    The "AffinityCall" implementation is a very simple, but less efficient implementation of the oshdb mapreducer: It's just sending separate affinityCalls() to the cluster for each data cell and reduces all results locally on the client.

    It's good for testing purposes and maybe a viable option for special circumstances where one knows beforehand that only few cells have to be iterated over (e.g. queries in a small area of interest), where the (~constant) overhead associated with the other methods might be larger than the (~linear) inefficiency with this implementation.

    See Also:
    Serialized Form
    • Method Detail

      • isCancelable

        public boolean isCancelable()
        Description copied from class: MapReducer
        Returns if the current backend can be canceled (e.g. in a query timeout).
        Overrides:
        isCancelable in class MapReducer<X>
      • isActive

        public boolean isActive()
      • mapReduceCellsOSMContribution

        protected <R,​S> S mapReduceCellsOSMContribution​(SerializableFunction<OSMContribution,​R> mapper,
                                                              SerializableSupplier<S> identitySupplier,
                                                              SerializableBiFunction<S,​R,​S> accumulator,
                                                              SerializableBinaryOperator<S> combiner)
                                                       throws Exception
        Description copied from class: MapReducer
        Generic map-reduce used by the `OSMContributionView`.

        The combination of the used types and identity/reducer functions must make "mathematical" sense:

        • the accumulator and combiner functions need to be associative,
        • values generated by the identitySupplier factory must be an identity for the combiner function: `combiner(identitySupplier(),x)` must be equal to `x`,
        • the combiner function must be compatible with the accumulator function: `combiner(u, accumulator(identitySupplier(), t)) == accumulator.apply(u, t)`

        Functionally, this interface is similar to Java11 Stream's reduce(identity,accumulator,combiner) interface.

        Specified by:
        mapReduceCellsOSMContribution in class MapReducer<X>
        Type Parameters:
        R - the data type returned by the `mapper` function
        S - the data type used to contain the "reduced" (intermediate and final) results
        Parameters:
        mapper - a function that's called for each `OSMContribution`
        identitySupplier - a factory function that returns a new starting value to reduce results into (e.g. when summing values, one needs to start at zero)
        accumulator - a function that takes a result from the `mapper` function (type <R>) and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter (mutate) the state of the accumulation value (e.g. directly adding new values to an existing Set object)
        combiner - a function that calculates the "sum" of two <S> values; this function must be pure (have no side effects), and is not allowed to alter the state of the two input objects it gets!
        Returns:
        the result of the map-reduce operation, the final result of the last call to the `combiner` function, after all `mapper` results have been aggregated (in the `accumulator` and `combiner` steps)
        Throws:
        Exception
      • flatMapReduceCellsOSMContributionGroupedById

        protected <R,​S> S flatMapReduceCellsOSMContributionGroupedById​(SerializableFunction<List<OSMContribution>,​Iterable<R>> mapper,
                                                                             SerializableSupplier<S> identitySupplier,
                                                                             SerializableBiFunction<S,​R,​S> accumulator,
                                                                             SerializableBinaryOperator<S> combiner)
                                                                      throws Exception
        Description copied from class: MapReducer
        Generic "flat" version of the map-reduce used by the `OSMContributionView`, with by-osm-id grouped input to the `mapper` function.

        Contrary to the "normal" map-reduce, the "flat" version adds the possibility to return any number of results in the `mapper` function. Additionally, this interface provides the `mapper` function with a list of all `OSMContribution`s of a particular OSM entity. This is used to do more complex analyses that require the full edit history of the respective OSM entities as input.

        The combination of the used types and identity/reducer functions must make "mathematical" sense:

        • the accumulator and combiner functions need to be associative,
        • values generated by the identitySupplier factory must be an identity for the combiner function: `combiner(identitySupplier(),x)` must be equal to `x`,
        • the combiner function must be compatible with the accumulator function: `combiner(u, accumulator(identitySupplier(), t)) == accumulator.apply(u, t)`

        Functionally, this interface is similar to Java11 Stream's reduce(identity,accumulator,combiner) interface.

        Specified by:
        flatMapReduceCellsOSMContributionGroupedById in class MapReducer<X>
        Type Parameters:
        R - the data type returned by the `mapper` function
        S - the data type used to contain the "reduced" (intermediate and final) results
        Parameters:
        mapper - a function that's called for all `OSMContribution`s of a particular OSM entity; returns a list of results (which can have any number of entries).
        identitySupplier - a factory function that returns a new starting value to reduce results into (e.g. when summing values, one needs to start at zero)
        accumulator - a function that takes a result from the `mapper` function (type <R>) and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter (mutate) the state of the accumulation value (e.g. directly adding new values to an existing Set object)
        combiner - a function that calculates the "sum" of two <S> values; this function must be pure (have no side effects), and is not allowed to alter the state of the two input objects it gets!
        Returns:
        the result of the map-reduce operation, the final result of the last call to the `combiner` function, after all `mapper` results have been aggregated (in the `accumulator` and `combiner` steps)
        Throws:
        Exception
      • mapReduceCellsOSMEntitySnapshot

        protected <R,​S> S mapReduceCellsOSMEntitySnapshot​(SerializableFunction<OSMEntitySnapshot,​R> mapper,
                                                                SerializableSupplier<S> identitySupplier,
                                                                SerializableBiFunction<S,​R,​S> accumulator,
                                                                SerializableBinaryOperator<S> combiner)
                                                         throws Exception
        Description copied from class: MapReducer
        Generic map-reduce used by the `OSMEntitySnapshotView`.

        The combination of the used types and identity/reducer functions must make "mathematical" sense:

        • the accumulator and combiner functions need to be associative,
        • values generated by the identitySupplier factory must be an identity for the combiner function: `combiner(identitySupplier(),x)` must be equal to `x`,
        • the combiner function must be compatible with the accumulator function: `combiner(u, accumulator(identitySupplier(), t)) == accumulator.apply(u, t)`

        Functionally, this interface is similar to Java11 Stream's reduce(identity,accumulator,combiner) interface.

        Specified by:
        mapReduceCellsOSMEntitySnapshot in class MapReducer<X>
        Type Parameters:
        R - the data type returned by the `mapper` function
        S - the data type used to contain the "reduced" (intermediate and final) results
        Parameters:
        mapper - a function that's called for each `OSMEntitySnapshot`
        identitySupplier - a factory function that returns a new starting value to reduce results into (e.g. when summing values, one needs to start at zero)
        accumulator - a function that takes a result from the `mapper` function (type <R>) and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter (mutate) the state of the accumulation value (e.g. directly adding new values to an existing Set object)
        combiner - a function that calculates the "sum" of two <S> values; this function must be pure (have no side effects), and is not allowed to alter the state of the two input objects it gets!
        Returns:
        the result of the map-reduce operation, the final result of the last call to the `combiner` function, after all `mapper` results have been aggregated (in the `accumulator` and `combiner` steps)
        Throws:
        Exception
      • flatMapReduceCellsOSMEntitySnapshotGroupedById

        protected <R,​S> S flatMapReduceCellsOSMEntitySnapshotGroupedById​(SerializableFunction<List<OSMEntitySnapshot>,​Iterable<R>> mapper,
                                                                               SerializableSupplier<S> identitySupplier,
                                                                               SerializableBiFunction<S,​R,​S> accumulator,
                                                                               SerializableBinaryOperator<S> combiner)
                                                                        throws Exception
        Description copied from class: MapReducer
        Generic "flat" version of the map-reduce used by the `OSMEntitySnapshotView`, with by-osm-id grouped input to the `mapper` function.

        Contrary to the "normal" map-reduce, the "flat" version adds the possibility to return any number of results in the `mapper` function. Additionally, this interface provides the `mapper` function with a list of all `OSMContribution`s of a particular OSM entity. This is used to do more complex analyses that require the full list of snapshots of the respective OSM entities as input.

        The combination of the used types and identity/reducer functions must make "mathematical" sense:

        • the accumulator and combiner functions need to be associative,
        • values generated by the identitySupplier factory must be an identity for the combiner function: `combiner(identitySupplier(),x)` must be equal to `x`,
        • the combiner function must be compatible with the accumulator function: `combiner(u, accumulator(identitySupplier(), t)) == accumulator.apply(u, t)`

        Functionally, this interface is similar to Java11 Stream's reduce(identity,accumulator,combiner) interface.

        Specified by:
        flatMapReduceCellsOSMEntitySnapshotGroupedById in class MapReducer<X>
        Type Parameters:
        R - the data type returned by the `mapper` function
        S - the data type used to contain the "reduced" (intermediate and final) results
        Parameters:
        mapper - a function that's called for all `OSMEntitySnapshot`s of a particular OSM entity; returns a list of results (which can have any number of entries)
        identitySupplier - a factory function that returns a new starting value to reduce results into (e.g. when summing values, one needs to start at zero)
        accumulator - a function that takes a result from the `mapper` function (type <R>) and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter (mutate) the state of the accumulation value (e.g. directly adding new values to an existing Set object)
        combiner - a function that calculates the "sum" of two <S> values; this function must be pure (have no side effects), and is not allowed to alter the state of the two input objects it gets!
        Returns:
        the result of the map-reduce operation, the final result of the last call to the `combiner` function, after all `mapper` results have been aggregated (in the `accumulator` and `combiner` steps)
        Throws:
        Exception
      • isActive

        public default <T> boolean isActive​(T ignored)
      • osmType

        @Deprecated(since="0.7.0",
                    forRemoval=true)
        @Contract(pure=true)
        public default M osmType​(OSMType type1,
                                 OSMType... otherTypes)
        Deprecated, for removal: This API element is subject to removal in a future version.
        use oshdb-filter filter(String) instead
        Limits the analysis to the given osm entity types.
        Parameters:
        type1 - the set of osm types to filter (e.g. `OSMType.NODE`)
        otherTypes - more osm types which should be analyzed
        Returns:
        `this` mapReducer (can be used to chain multiple commands together)