Question:When I run below command in Databrick , I get output where I have three columns called name, id and age but when I try to filter on name by running below I get below error as Name column do not exist. What wrong am i doing?
Now I do filter oon Name like this
Answer:Filtering happens before your expand your array of structs. You have two choices here:
- Use common table expressions to explode first & then filter:
- Use the filter function to filter out data inside the array with something like that (not tested), but it may require doing the filtering two times:
If you have better answer, please add a comment about this, thank you!