Module 5- Exploring and Manipulating Data

Figures 1-4: Creating a File geodatabase and copying data into it.

Flow chart denoting the cradle to grave process of creating a file geodatabase, copying data, and using cursors to retrieve and populate a data dictionary.

This week, we dove into manipulating data using Python. We learned the ease of copying data to geodatabases, creating a search cursor to extract data from said geodatabase, and then creating and populating an empty dictionary.

 

Steps 1-4

 

The first four steps for creating this script were straightforward and had no issues copying the data into the geodatabase. The issue I ran into was when I went back in to add print statements in the for loop for each feature class, I accidentally had all the print statements say “create feature class” instead of listing the desired fields. It was not an easy fix. I had to restart Arc GIS and hope for the best.

 

For Step 3: When using the search cursor, the delimiter function needed to be used. This is what caused the most confusion because I thought I could just use cursor = arcpy.SearchCursor(fc) the and get the results back. But then I realized I had to use the delimitedField = arcpy.AddFieldDelimiters() and this narrowed down the search to the FEATURE class.

Finally, I had to ensure to turn the population field, which was an integer, to a string in the get value field because Python cannot concatenate integers, only strings.

 Step 5

Creating the dictionary deemed to be the most difficult task of all. I could not figure out why the for loop and the chosen function to add pairs of items to a dictionary was printing out an empty dictionary. Finally, after meeting at office hours, I realized that I had to create a second search cursor for the dictionary creation and add the population plus the city names as parameters. Additionally, although dictionaries cannot be indexed, you do need to start as row [0] to start populating values in. This was a key part of the dictionary’s creation.

 

Module 4 - Geoprocessing

 

                        Results of Geoprocessing operations; Add XY, Buffer and Dissolve

In this weeks module, we finally got our hands on the creation of script ad connecting it to our Map View in Arc GIS pro. what first comes to mind when I ran model builder and wrote script for common geoprocessing operations, was "I wish I knew all of this durin Intro to GIS class!" 

A key take away form this week's lab was that there are multiple ways to get to the same results. For example, in our script, we could have assigned variables to the different parts of syntax for our geoprocessing operations and make our script more flexible instead of using file paths. 

Additionlly, we could have also created a moel in model builder for both the add XY operation and buffer and used this to export a puthon script. 

Ultimately, this versatile nature of python makes me ecited to learn different inputs that could take me to the same desired result. 

                           ___________________________________________________

For more pointer on the Add XY , buffer, and dissolve operation, please see notes below. 

Part 2 - Ad XY:

 

Remember to import the environment and workspace.

To make script more flexible, it’s best practice to use variables instead of the hard code in the script.

So, I assigned the hospitals shapefile to the in_features argument in the ADDXY function.

Ensure you to add (env.overwriteOutput = True) to override existing files.

 

 

Part 3 - Creating a 1000 Meter buffer and Dissolve:

 

Use the buffer analysis tool syntax to create a code block that performs the 1000-meter buffer. The arc GIS pro help page was a life saver when trying to resolve this as it gave a description of what was required for each parameter in the function.  It was important to remember to enter( “ ” )for optional empty parameters. Lastly, it was important to choose the correct dissolve option, ALL, to ensure all buffers were dissolved into a single feature class.

 

Module 3 - Debugging and Error Handling

Figure 1: Results of Script 1

Figure 2: Results of Script 2

Figure 3: Results of Script 3

Figure 4: Flowchart for Script 3 - This flowchart denotes the process for creating the try-except statement in script three that allowed the full script to run regardless of existing errors. 

Module 3 introduced error handling in our scripts. Knowing about common errors and how to troubleshoot is essential to saving time while coding. A type of error, called exceptions, are errors that occur during run time and wreak havoc on your operation. A solution to these unexpected errors is using a "try except" statement that allows the script to run while exceptions are trapped and "handled". Part 3 of our assignment, denoted below, shows the process for using the try-except method for script 3. 

 Part 3: Make a code run even with errors

The try-except statement process was not nearly as tedious or as terrifying as previously imagined. When the first error occurred, a type error, I proceeded to write a try-except statement for that section. When that ran, another error came up, a name error. Then, I wrote another try except for that! After running that, another type error appeared! Yikes! Then it hit me; you can do an all-encompassing try-except statement for the whole block of code using the catch-all Exception class. The statement began right after defining aprx and right before the final print statement was processed.  

I had to recall the importance of indentation so that the try except statement recognized the complete block of code.

Once the code was run, part b ran with no issue. 

Overall, I really enjoyed this section of the class and see how useful this will become in the future. 

Module 2 - Python Fundamentals

 

Results of the Module 2 Script. 

 

Flowchart for Creation of the Module 2 Script

GIS programming has proven to be the utmost challenge yet! We are here to rise to the challenge. This week introduced us to GIS basics, from syntax to while and for loops; we dove in deep. 

The initial steps of the assignment were fairly intuitive; it was not until step 3 that the fun really began. A description of my processes for steps 3 and 4 is seen below. 

Step 3: Create a loop that adds 20 random numbers between zero (0) and ten (10) to a list

 

The first step was to import the random module because the instructions specifically asked for a random number. **Like rolling dice.

Assign the lucky list to be empty so that it can be filled in later.

Next is the while loop-

Set counter as indicated, start at zero,

We used the append function because that would instruct us to add the numbers to the empty luckyList. 

We set the while i < 20 because this is how many iterations we want until the loop equals false and stops

Then we used the append method to add the random integers, from ranges 0-10, to the empty list.

We really did not need to, but the instructions asked us to, so we included an if and break statement that would prevent the list from going further than 20 iterations.

 

Then we hit print luckyList so that we could print out a list as requested.

 

***Using a for loop would have been a better, more concise option.

 

Step 4: Create a loop that removes an unlucky number from the list generated in step 3.

 

For this step, I began by copying luckyList, generated in step 3, into the cell.

Then an integer is assigned to the variable x.

Created an if statement where if it was not equal to x, it would say that it was not in the loop. However, if it were, the else statement could invoke the count method to count how many times that integer occurred in the list

Then, creating a print statement that mentioned how many times a specific number was to be removed was rather confusing, and I had to do some external research. I first had to write out the statement with parentheses and variables within, for example, (x) and (count). I had to write a count assignment statement just like I did for x.

Finally, the creation of the while loop using the remove method was used to remove the unlucky variable x from the previously generated list.  I struggled with the creation of this loop because I did not write the remove method on the same line that mentioned the lucky list. Therefore, every time I tried to print the while loop, an error would appear saying that x did not appear in the list.

Finally, the updated luckyList with no variable x was printed.

 

Module 1 - Flowcharts and IDLE

Answer:171.89 Degrees

 Figure 1. Flowchart depicting Conversion of Radians to Degrees

 

   Process Summary

Part 1: Using a Python Script to Organize the S Drive Folders.

 In the Intro to GIS course, a crucial step was to organize our digital folders and filepaths; this was typically a manual, zip, copy, and paste operation. Over time, it became second nature. However, in this course, we were introduced to the concept of automatically copying folders from the repository into our S drive using a premade script that was copied and executed into IDLE. This will make the digital fordel organization endeavors efficient and straightforward.  

Steps-

Identified the GIS Programming folder in the repository.

Copied the corresponding script to the S drive root.

Opened Windows icon on the left-hand corner and typed in “python command prompt”.

 Clicked open

Typed in IDLE at the end of the prompt to launch the interactive environment

Clicked on new file, opened at the top left-hand corner, then navigated to the S drive and clicked on the pre-written script.

Clicked on the “run” at the top of the IDLE environment- Confirmed execution.

To verify, open the S drive and find the GISProgramming folder and all its corresponding modules in the drive.

 

Other Thoughts-

            This was challenging as the Intro to GIS class drilled into me the zip file, copy and paste technique to organize folders.

 

I ran the script a few times before I finally succeeded. What caused issues was that I was not sure “what root of your S:\ drive.” was. Then I realized it was literally just copy and paste into the S drive. Then, I had forgotten that I had to click “run” at the top of the IDLE for the script to run. These were all silly simple mistakes. This method of copying the folder from the R drive is such a time saver. I look forward to using this again.

 

Over all, opening python, opening IDLE, ect all seems to be very straightforward and almost naturally intuitive.

 

Part 2: Flowcharting: Convert 3 radians to degrees

                        Steps

                        Establish Formula: degrees = radians*180/pi

Write Pseudocode: Remember indentation, starts, stops, and which steps are dependent on others. **Remember to write the 3 as 3.0

 

#This program converts radians to degrees

Start

Radians = 3.0

pi = 3.14159

Degrees = radians * 180 / pi

Print Degrees

End

 

                      Manually check calculation is correct.

                                    Use pseudocode to create a flowchart.

Open app.diagram.net

                                    Use Standardized symbols snip from the Python Scripting for ARCGIS pro as                                        reference.

                                    Save and export flowchart as png file  

 

 

Other Notes

 

I was beginning to think about the term’s “script” and “source code” as interchangeable, but they are not. I had to do a little side quest to find the difference.

Source Code- Written in programing language. Compiled.

Script- sort of like a stepchild to the program code. Interpreted, not compiled.

 

Useful Websites for more information on GIS Notebooks and IDLE

https://pro.arcgis.com/en/pro-app/latest/arcpy/get-started/pro-notebooks.htm

https://gis.stackexchange.com/questions/225585/differences-between-python-window-and-idle-for-scripting-with-arcgis-desktop

            **Feedback from the GIS community.

 

Overall, the most difficult task during this lab was deconstructing the individual processes that we all know, such as a simple equation, into individual steps that make sense, are sequential, all while following standard symbols. 

 Zen of Python

The first thought that comes to mind is that “The Zen of Python” is not about Python at all but rather a deep insight into the soul; but of course, it can’t be, or can it?

Despite my eagerness to dig deeper into the connotation of this text, I believe the author is trying to convey that…

A)    Python is straightforward.

B)     The process should not be incredibly complex.

C) Be sure to take the time to revise and weed out errors.

D)    To not worry, if you’re persistent, you will find a solution to the pickle you find yourself in.

Additionally, lines 3 to 7 might refer to the imputed code's structure; it should not be dense, simple, straight to the point, neat, etc.

There are several lines, specifically towards the end of the passage, that nod to the idea of “don’t overcomplicate it, the solution should be simple and odds are that this is not the first time it happens”.

This is in theme with the whole branding of Python; an easy-to-use and learn, fast, efficient technology that is here to make daunting tasks better.

 Ultimately, the reader is encouraged to find solace in the fact that the world of Python is only as difficult as they make it. These are my first thoughts, might also be completely wrong, but I digress.

Overall, a very interesting passage. I have come to realize that the tech community has a lot more fun with their naming conventions, and now also writing passages, than I had previously thought. 

Final Project- Bobwhite Manatee Transmission Line Analysis

 

Multiple Map outputs for the Bobwhite Manatee Analysis

It is finally here, the final project!

After learning new critical skills in this course, we were let loose to show them off in an "after the fact" analysis of the Bobwhite Manatee Transmission line to see its impact on schools, environmentally sensitive areas, and homes/parcels.

With objectives clearly defined, and previous lab material as reference, I was able successfully complete the mission and product maps and a presentation I am proud of.

To see my work, please check out my ArcGIS Story Map and presentation in the links below.

Story Map- https://arcg.is/11XOHH0

Video Presentation- https://drive.google.com/file/d/1y0_SUdu7Cg0SZAwjm52d_KqDadpu1eAk/view?usp=drive_link

Video Transcript- https://docs.google.com/document/d/1xJQFKLyQI2oCgxCS_9h_jYi3Ky5IDA4V/edit?usp=drive_link&ouid=104066649745176126190&rtpof=true&sd=true

Module 6 - Georefferencing

 

Map portraying the FWC conservation easements for the bald eagle's next on the UWF campus.
Inset maps for easier observation of easements and the location of the nest in the state of Florida provided.

Module 6! What a blast! This module was by far the most comprehensive and relatable module. Real life application was utmost in this lab. 

Georeferencing is the art of assigning geographic location to a raster data set, i.e. an arial picture.

As one might assume, differences in image size, distortion, etc., could make the process quite challenging. 

Using a vector polygon layer of UWF buildings with a known coordinate system, we were able to use georeferencing tools to align or superimpose the raster dataset to the polygons and thus assign a projected coordinate system. It takes trial and error, zooming in an out of images to get points to align; but overall, once you get a good number of controls points with a low total root mean square error value, you will have aligned your image as closely as possible to their actual points on earth. 

We were able to experiment with the difference in transformation orders, and how overall appearance, not just a low RMS value, means higher map accuracy; higher order transformations allow for more bending of the raster data. 

We proceeded to georefference a real-life parcel survey, where the image had no building references, and we had to use other cues and reference points to align. This was trickier than the first two digital images as proportions were completely different and the lack of buildings as references was a roadblock. This is similar to how a real-life parcel developer would draw out plans and superimpose them over existing landscape when creating their development proposal. 

Furthermore, we used the editing tools do create vector features, a building, and a road features, as one would do when planning development in an area or denoting features. 

Lastly, we took this all full-circle and used our buffer tools to denote the bald eagle's nest in reference the UWF campus and the conservation easements zones established by the Florida Wildlife Commission to ensure their nest was protected. We even hyperlinked an image of the nest to our eagle's nest point feature attributes so that the end user could retrieve a picture!

After solving problems, ensuring eagles were protected, and relating images to their earthly coordinates, it was a good interactive week to gain more real-life skills in GIS!