Blog

We've just released the latest iteration of our NYC Mass Transit Spatial Layers series (Jan 2017), which contain a number of important updates.

  1. We've completely re-organized the files for MTA bus stops and routes. Instead of having separate borough files for routes and stops (including the confusing mixture of bus company and local Queens bus routes), we have aggregated all of the local buses into one city-wide file, and have separated the express bus services into their own dedicated file.

  2. The new MTA subway stops file includes the three new stations on the 2nd Ave subway. There's also a new station on the SIR that replaces two older stations that have been demolished. The new routes file includes the W train and the extension of the Q train up 2nd Ave. In some cases we inserted the new routes data ourselves, as the MTA static feeds that were released in January still lacked this information. The train attribute of the stops file has been updated to indicate which trains stop there. There have been no updates to the subway entrances file, so we're keeping the previous one from spring 2016.

  3. We've added stops and routes files for the PATH train (Port Authority Trans-Hudson) which is the rapid transit system that connects Newark, Jersey City, and Hoboken to Manhattan. The files are a subset and derivative of data produced by the NJ GIS Office, which in turn were derived from GTFS schedule data from NJ Transit. We pulled out just the PATH train and have re-projected it to NY State Plane Long Island so that it fits seamlessly with the MTA mass transit layers in the series.

As usual we've moved our previous series (May 2016) into the NYC Mass Transit Spatial Layers Archive. Over the summer we hope to harmonize the subway ridership data that we include in the NYC Geodatabase with these layers.

 

 

Here are the latest updates in the new version (Jan 2017):

  • All of the American Community Survey data tables for PUMAs, ZCTAs, and census tracts have been updated with new data from the 5-year 2011-2015 ACS (the previous version of the database had tables from 2010-2014). The Census Bureau has re-coded all of their variables for asked rent, so that the bottom and top values have increased to reflect an increase in prices. The new variables do a better job at representing the distribution / spread of rent values.

  • We updated the subway stations layer from the latest MTA data feeds. The new stations layer includes: the three new stations on the 2nd Ave subway line, the new station on the Staten Island Railway (and the removal of two stations that the new one replaced), and updated train attributes to account for the re-routing of the Q train and the re-introduction of the W train. The related subway complexes layer was not updated, as new ridership data isn't available for these stations yet (the complexes layer represents the state of subway service and ridership for 2015). That update will come this summer.

All metadata and documentation is up to date. If you still need data from the previous version (Aug 2015) it's been moved to the NYC Geodatabase Archive section.

Registration is now open for the spring semester’s GIS (geographic information systems) Practicum, Introduction to GIS Using Open Source Software (featuring QGIS). There will only be one session this term, held in the GIS Lab at Baruch College:

  • Friday Mar 10th

The day-long workshop runs from 9am to 4:30pm. Current CUNY graduate students, faculty, and staff, and full-time Baruch undergrads are eligible to register. Advance registration is required; the fee is $30 and includes a detailed tutorial manual and a light breakfast. Participants must bring their own laptop with QGIS pre-installed in order to take the class. Visit the GIS Practicum page to learn more and to register: http://guides.newman.baruch.cuny.edu/gis/gisprac.

Registration is now open for the fall semester’s GIS (geographic information systems) Practicum, Introduction to GIS Using Open Source Software (featuring QGIS). The sessions will be held in the GIS Lab at Baruch College:

  •     Friday Sept 30th
  •     Friday Oct 28th
  •     Friday Nov 18th

The day-long workshop runs from 9am to 4:30pm. Current CUNY graduate students, faculty, and staff, and full-time Baruch undergrads are eligible to register. Advance registration is required; the fee is $30 and includes a detailed tutorial manual and a light breakfast. Participants must bring their own laptop with QGIS pre-installed in order to take the class. Visit the GIS Practicum page to learn more and to register: http://guides.newman.baruch.cuny.edu/gis/gisprac.

It may be summer, but the GIS Lab is still churning away! I have three updates to share.

First, the latest version of the GIS Practicum manual is now available. The workbook has been updated using the latest long term release of QGIS, 2.14 Essen. While most of the revisions are cosmetic tweaks to reflect changes in the interface, the sections on raster data and web mapping services did get some notable additions. We'll be running workshops using the new material this coming fall. CUNY affiliates who are interested in being notified once registration opens can sign up here.

Second, we've released the latest version of the NYC Geodatabase. There are two big updates here. First, the latest ZIP Code Business Patterns data has been added, with 2014 data replacing 2013. Second, all the subway data has been updated, with 2015 ridership data for the MTA and the PATH stations that are in NYC. As always, you can choose between a SQLite version that's optimal for QGIS or the Spatialite GUI or CLI tools, and an MS Access personal geodatabase that's suited for ArcGIS. 

Lastly, I've been updating many of the library's research guides and some of my tutorials. In the GIS Guide I've revamped the web mapping section, tossing the old Google map stuff and replacing it with videos and links about CARTO. In conjunction with these changes, I've updated several Tutorials including the one on bringing data into CARTO, as they've recently changed their name (from CartoDB) and interface. The NYC data guide has been cleaned up by adding several new resources and tossing some old ones. Also made some minor edits to the guides on US Census Data and Demography; for the latter, check out all the new book's we've purchased under New Titles.

This is an important update regarding several of the bus stop and route features in our NYC Mass Transit Spatial Layers series. We've discovered some anomalies between our descriptions and the file contents, and have updated our metadata accordingly:

  • The bus route and stop files initially designated for Queens actually just contain features in eastern Queens, primarily east of Flushing Meadows and I-678. We've changed the titles of these files to Eastern Queens and updated the descriptions accordingly.

  • The local bus routes and stops for western Queens are lumped into the Bus Company (i.e. express bus) files for the entire city. We've changed the titles of these files to Western Queens and Bus Company and updated the descriptions accordingly.

This isn't an ideal arrangement, as the local buses in Queens are fundamentally different from the express buses. You can distinguish between the local and express services by looking at the route number in the attribute table: if the route is designated with a 'Q' immediately followed by a number, it's a local Queens route. Otherwise, it's an express bus route. It's harder to tell with the stops; you can compare them against the routes to see which are local and which aren't. Most of the stops located in western Queens will be local bus stops.

Why are the files structured this way? It has to do with the internal organization of the MTA and how the buses are managed. The bus lines are divided into different groups based on how they are administered, and the data is structured to reflect this. When we download the raw GTFS bus files to process them, we download one set designated for each borough and one set designated for the bus company. Whatever is in those original sets gets carried over into the new files we create. Our files simply follow the provenance of the MTA files.

In some cases it's normal for bus routes or stops to appear in a neighboring borough, and we've noted this in our metadata all along. For example, the M60 is a Manhattan local that runs through Harlem to LaGuardia Airport In Queens. It appears in the Manhattan files and not the Queens files since it's designated as a Manhattan bus and stops primarily in Manhattan. Other cases are odd exceptions; the M100 runs in northern Manhattan from Harlem to Inwood, but is included in the Bronx files and not the Manhattan ones. In that case, it's simply because that route is managed / administered alongside the Bronx routes.

But the split within Queens and the combination with the express buses is counter-intuitive, and we're going to consider re-organizing this data when we assemble our next version at the end of this year. For now,  we've updated the metadata for the May 2016 version so that users are aware of this issue. If you want all bus stops or routes for Queens, this data is split into two files: a file with local buses for eastern Queens, and another with local buses for western Queens that also includes express buses city-wide.

 

 

It's been a busy semester, and now that it's ending we've wrapped up many of our projects and are ready to share the results. As always, we've meticulously documented all of our datasets so you know exactly what you're getting.

  • NYC Mass Transit Spatial Layers - Janine has created the latest version of our NYC bus, train, and subway features that we process and assemble from the MTA's raw data feed. If you want to make some nice subway maps or need to figure out where all the buses are going, look no further!

  • NYC Geocoded Real Estate Sales - Anastasia has completed the gargantuan task of creating this new dataset, where she's aggregated all of the city's real estate sales from 2003 to 2015 AND geocoded them using the city's geocoding API. We even went the extra mile and manually identified all unmatched records so that we have a complete dataset. We're making the layers available as shapefiles for each year, and as one big collection in a Spatialite database.

  • US Census Geocoding Script - I've written a Python 3.x script that uses the Census Bureau's Geocoding API and the external censusgeocoder module to batch process delimited text files of parsed and unparsed addresses. Check out the documentation that's included with the script for details.

Even though the summer is here, we'll keep plugging away - check the GIS Lab for our availability. Later this summer a new version of the NYC Geodatabase will be rolled out, and the latest datasets I've mentioned here will also be ported over to the NYU's spatial repository as part of our new and exciting collaboration. We have a few other new datasets in the works too, so stay tuned.

 

Registration is now open for the spring semester’s GIS (geographic information systems) Practicum, Introduction to GIS Using Open Source Software (featuring QGIS). The March workshop will be held at the Department of Earth, Environmental, and Geospatial Sciences at Lehman College in the Bronx (20 seats, PCs provided). The April session will be held at Baruch in the GIS Lab (10 seats, participants must bring laptops).

  • Friday Mar 4th (at Lehman)
  • Friday Apr 1st (at Baruch)


The day-long workshop runs from 9am to 4:30pm. Current CUNY graduate students, faculty, and staff, and full-time Baruch and Lehman undergrads are eligible to register. Advance registration is required; the fee is $30 and includes a detailed tutorial manual and a light breakfast. Visit the GIS Practicum page to learn more and to register: http://guides.newman.baruch.cuny.edu/gis/gisprac.

This semester we're also bringing back the Spatial Databases workshop (featuring Spatialite). The half-day workshop will run from 9am to 12:30pm on Friday April 8th at Lehman College. Eligibility requirements are the same, except that participants must also have prior GIS experience as this is a more advanced workshop. The registration fee is $5. Visit the Spatial Database Practicum page to learn more and to register: http://guides.newman.baruch.cuny.edu/gis/spatialdb.

We've just released the latest version of the NYC Geodatabase, a resource designed for mapping and analyzing NYC city-level features and data in GIS. We're continuing to offer two formats that have identical content - a Spatialite version for QGIS and Spatialite users, and an MS Access personal geodatabase for ArcGIS users (although the Spatialite version can also be used in ArcGIS from version 10.2 forward).

Here are the latest updates in the new version (Jan 2016):

  • All of the American Community Survey data tables for PUMAs, ZCTAs, and census tracts have been updated with new data from the 5-year 2010-2014 ACS (the previous version of the database had tables from 2009-2013).

  • We made a minor update to the subway stations layer, to include the new station on the 7 line at 34th St - 11 Av (Hudson Yards). We did not update the related subway complexes layer, as ridership data for 2015 (which would include this new station) isn't available yet. Those updates will come with the next version this summer.

All metadata and documentation is up to date. If you still need data from the previous version (Aug 2015) it's been moved to the NYC Geodatabase Archive section.

 

 

We've just released the latest version of the NYC Mass Transit Spatial Layers series. The data are sourced from the MTA's transit feed and are processed to represent usable, public, and well-documented GIS layers for representing New York City's bus, subway, and train routes and stations. The files are in a shapefile format and are projected in NY State Plane Long Island (US feet).

Here are the major updates to this new version (Nov 2015):

  • The underlying stops and routes data for the subway include the city's newest subway station on the 7 line, 34th St - 11 Ave (Hudson Yards). The underlying data for subway entrances did not include the new station, so we added it to the file.
  • There were small fluctuations in the number of bus stops in each borough between this iteration and our last one (in May 2015), and a correction was made to one of the bus routes in northern Manhattan, to extend the route to it's final stop (the route fell short off the stop in the previous version).
  • The routes file for the Metro North remains highly generalized and is only appropriate for schematic representation at the metropolitan-area level, but we did make minor improvements to the line work so that it is better than our previous version.

All of the metadata for the layers has been updated. Data from the previous iteration of the files (May 2015) has been moved to a new archives page and will remain accessible.

Here are the latest updates in the new version (Aug 2015):

  • All of the NYC facility point features - colleges, hospitals, libraries, and schools - have been updated. The new features are derived from the 2015 version of city planning's programs and facilities database; the old features were from 2013.
  • The subway stations layer has been recently rebuilt from the MTA's transit data feed, and the subway complexes layer has been updated with the latest ridership data for 2014.
  • The previous PATH train stations layer for NYC was dropped in favor of a new source from NJ Transit. We've also added ridership data for these stations from the NY & NJ Port Authority from 2012 to 2014.
  • We've swapped out the 2012 ZIP Code Business Patterns with new data for 2013. Data on employment, establishments, and wages is aggregated to the ZCTA level.

All metadata and documentation is up to date. If you still need data from the previous version (Jan 2015) it's been moved to the NYC Geodatabase Archive section.

Greetings! With this new version of the Baruch Geoportal, launched July 30, 2015, we hope to provide you (our users) with a simpler way of navigating and accessing our content. The wiki-style interface should also make the job of our geospatial services team easier, so we can focus more on the content and can make updates more quickly.  Most of the content that was housed on the old geoportal (http://www.baruch.cuny.edu/geoportal/) has been migrated to this new location (https://www.baruch.cuny.edu/confluence/display/geoportal/).

A few recent updates that I'll mention:

  • The latest version of the GIS Practicum manual, Introduction to GIS Using Open Source Software, has just been released. The manual was written using QGIS 2.8 Wien, which is the current long term service version of QGIS. We'll be using this manual in our workshops for the 2015-16 Academic Year.
  • Our latest spatial dataset is the NYC Mass Transit Spatial layers series, which was created to provide the public with free and well-documented GIS layers for NYC transit features. It includes routes and stops for subways, buses, and trains.
  • We've consolidated a number of resources that were previously scattered on the GIS research guide and have created dedicated places for them here. This includes geographical crosswalks and maps for relating various NYC geographies, and data files with individual records for every IRS Tax Exempt organization in NYC.
  • We have an About US page that discusses our mission and our services
  • Whenever possible we've provided a direct link to geospatial metadata for our data files in an XML format. Over the coming months we'll be developing style sheets to make these files human-readable.
  • The NYC Geodatabase is the next update on our list! We hope to have a new version posted before the end of August.


Feel free to follow this blog to keep up to date with additions and changes to our datasets. If you are a CUNY student, faculty, or staff member who has created GIS data as part of your coursework or research, please contact us if you would like to host your data on this site. The geospatial services team will assist you in preparing your data and in creating standards-based metadata for your datasets.